PROYECTO DE CLASE - ANÁLISIS DE SERIES DE TIEMPO

SERIE: ANALISIS TEMPORAL DE IMPORTACIONES

Autor:

  • Cesar Prieto

INTRODUCCION

Esta serie consta del valor FOB en dólares de las importaciones que llegan a los puertos de Colombia vía marítima y su destino final es la ciudad de Bogotá, en el periodo transcurrido entre enero del 2012 hasta diciembre de 2021, la serie es de tipo mensual.

Definición: El valor FOB en dólares de la mercancía, es valor de la mercancía en el momento que se carga a bordo del medio de transporte, en este caso el marítimo.

La serie consta de 120 observaciones, lo que corresponde a los 10 años transcurridos desde el 2012 hasta el 2021

Descripcion de la base:

  • Importaciones MENSUALES via maritima con destino a Bogotá
  • 3401 Datos tomado (Desde el 01/2012 Hasta 12/2021)
  • Variable a usar: VAFODO

PRIMER ANALISIS Y DESCRIPCION DE LA SERIE

IMPORTACION Y LIMPIEZA DE DATOS

In [4]:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import statsmodels.api as sm
import plotly.express as px
from sklearn import __version__ as sklearn_version
from sklearn import *

import os
In [5]:
import warnings
warnings.filterwarnings("ignore")
In [6]:
current_dir = os.getcwd()
pro_series_dir = os.path.abspath(os.path.join(current_dir, '..', '..', '..'))
pro_series_dir
Out[6]:
'C:\\Users\\Cesar Prieto\\Documents\\REPOS GIT\\Pro-Series\\Pro-Series'
In [7]:
# Construir la ruta a la carpeta de Datos
data_folder_path = os.path.join(pro_series_dir, 'Datos')
data_folder_path = data_folder_path + os.sep
data_folder_path
Out[7]:
'C:\\Users\\Cesar Prieto\\Documents\\REPOS GIT\\Pro-Series\\Pro-Series\\Datos\\'
In [8]:
# Cargar el archivo CSV en un DataFrame de pandas
Importaciones = pd.read_csv(data_folder_path + "Importaciones.csv", sep=',')
In [9]:
Importaciones.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 144 entries, 0 to 143
Data columns (total 13 columns):
 #   Column          Non-Null Count  Dtype  
---  ------          --------------  -----  
 0   FECH            143 non-null    float64
 1   VACIP           144 non-null    int64  
 2   VAFODO          144 non-null    float64
 3   FLETE           144 non-null    float64
 4   IMP1            144 non-null    float64
 5   VACID           144 non-null    float64
 6   PBK             144 non-null    float64
 7   PNK             144 non-null    float64
 8   CANU            81 non-null     float64
 9   fecha           144 non-null    object 
 10  mes             144 non-null    object 
 11  año             144 non-null    int64  
 12  fecha_completa  144 non-null    object 
dtypes: float64(8), int64(2), object(3)
memory usage: 14.8+ KB
In [10]:
#Exploracion de las variables encontradas en la base
Variables = Importaciones[['fecha_completa', 'VACIP', 'VAFODO', 'FLETE', 'IMP1', 'VACID', 'PBK', 'PNK']].copy()

# Convertir la columna 'fecha_completa' a formato de fecha
Variables['fecha_completa'] = pd.to_datetime(Variables['fecha_completa'])

Variables.info()
Variables.head(5)

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 144 entries, 0 to 143
Data columns (total 8 columns):
 #   Column          Non-Null Count  Dtype         
---  ------          --------------  -----         
 0   fecha_completa  144 non-null    datetime64[ns]
 1   VACIP           144 non-null    int64         
 2   VAFODO          144 non-null    float64       
 3   FLETE           144 non-null    float64       
 4   IMP1            144 non-null    float64       
 5   VACID           144 non-null    float64       
 6   PBK             144 non-null    float64       
 7   PNK             144 non-null    float64       
dtypes: datetime64[ns](1), float64(6), int64(1)
memory usage: 9.1 KB
Out[10]:
fecha_completa VACIP VAFODO FLETE IMP1 VACID PBK PNK
0 2012-01-01 869285742501 1.579419e+10 1.039715e+11 9.063473e+10 2.243658e+12 2.727896e+10 5.725524e+09
1 2012-02-01 753291179381 1.787081e+10 5.944155e+10 8.521563e+10 1.327547e+12 6.703622e+09 1.794043e+10
2 2012-03-01 970221001405 1.969805e+10 8.800024e+10 1.190929e+11 1.990896e+12 2.428291e+10 5.363545e+09
3 2012-04-01 829378952699 1.597839e+10 8.199698e+10 1.523165e+11 1.724013e+12 1.018825e+10 5.578638e+09
4 2012-05-01 1041614618811 2.355538e+10 1.157984e+11 1.217380e+11 2.454812e+12 2.360439e+10 6.344622e+09
In [11]:
Variables[['VACIP', 'VAFODO', 'FLETE', 'IMP1', 'VACID', 'PBK', 'PNK']] = Variables[['VACIP', 'VAFODO', 'FLETE', 'IMP1', 'VACID', 'PBK', 'PNK']].div(1000000000)
Variables.head(5)
Out[11]:
fecha_completa VACIP VAFODO FLETE IMP1 VACID PBK PNK
0 2012-01-01 869.285743 15.794190 103.971527 90.634731 2243.657871 27.278956 5.725524
1 2012-02-01 753.291179 17.870806 59.441549 85.215632 1327.547259 6.703622 17.940430
2 2012-03-01 970.221001 19.698045 88.000244 119.092896 1990.895960 24.282915 5.363545
3 2012-04-01 829.378953 15.978388 81.996980 152.316541 1724.013363 10.188252 5.578638
4 2012-05-01 1041.614619 23.555380 115.798390 121.737966 2454.811904 23.604388 6.344622
In [12]:
import plotly.graph_objects as go

# Crear una figura de plotly
fig = go.Figure()

# Agregar cada línea al gráfico
fig.add_trace(go.Scatter(x=Variables['fecha_completa'], y=Variables['VACIP'], mode='lines', name='VACIP'))
fig.add_trace(go.Scatter(x=Variables['fecha_completa'], y=Variables['VAFODO'], mode='lines', name='VAFODO'))
fig.add_trace(go.Scatter(x=Variables['fecha_completa'], y=Variables['FLETE'], mode='lines', name='FLETE'))
fig.add_trace(go.Scatter(x=Variables['fecha_completa'], y=Variables['IMP1'], mode='lines', name='IMP1'))
fig.add_trace(go.Scatter(x=Variables['fecha_completa'], y=Variables['VACID'], mode='lines', name='VACID'))
fig.add_trace(go.Scatter(x=Variables['fecha_completa'], y=Variables['PBK'], mode='lines', name='PBK'))
fig.add_trace(go.Scatter(x=Variables['fecha_completa'], y=Variables['PNK'], mode='lines', name='PNK'))

# Actualizar el diseño del botón
fig.update_layout(
    updatemenus=[
        dict(
            buttons=list([
                dict(label="VACIP",
                     method="update",
                     args=[{"visible": [True, False, False, False, False, False, False]},
                           {"title": "Serie de tiempo variable VACIP"}]),
                dict(label="VAFODO",
                     method="update",
                     args=[{"visible": [False, True, False, False, False, False, False]},
                           {"title": "Serie de tiempo variable VAFODO"}]),
                dict(label="FLETE",
                     method="update",
                     args=[{"visible": [False, False, True, False, False, False, False]},
                           {"title": "Serie de tiempo variable FLETE"}]),
                dict(label="IMP1",
                     method="update",
                     args=[{"visible": [False, False, False, True, False, False, False]},
                           {"title": "Serie de tiempo variable IMP1"}]),
                dict(label="VACID",
                     method="update",
                     args=[{"visible": [False, False, False, False, True, False, False]},
                           {"title": "Serie de tiempo variable VACID"}]),
                dict(label="PBK",
                     method="update",
                     args=[{"visible": [False, False, False, False, False, True, False]},
                           {"title": "Serie de tiempo variable PBK"}]),
                dict(label="PNK",
                     method="update",
                     args=[{"visible": [False, False, False, False, False, False, True]},
                           {"title": "Serie de tiempo variable PNK"}])
            ]),
            direction="down",
            pad={"r": 10, "t": 10},
            showactive=True,
            x=0.05,
            xanchor="left",
            y=1.15,
            yanchor="top"
        ),
    ]
)

# Actualizar el diseño del eje x
fig.update_xaxes(title_text='Fecha', rangeslider_visible=True)

# Actualizar el diseño del eje y
fig.update_yaxes(title_text='Valor')

# Actualizar el título del gráfico
fig.update_layout(title='Series de tiempo de variables')

# Mostrar el gráfico
fig.show()

ELECCION DE LA VARIABLE Y DESARROLLO DE METODOLOGIAS

En la parte anterior del codigo pudimos ver como todas las variables poseen un comportamiento bastante diferente. Por esto hemos decidido que la variable elegida para desarrollar el proyecto sera la VAFODO o mejor dicho el Valor FOB dólares de la mercancía. Valor de la mercancia en el momento que se carga a bordo del medio de transporte marítimo.

In [13]:
# Seleccionar solo las columnas 'fecha_completa' y 'VAFODO'
Serie = Variables[['fecha_completa', 'VAFODO']]
print(Serie)

    fecha_completa      VAFODO
0       2012-01-01   15.794190
1       2012-02-01   17.870806
2       2012-03-01   19.698045
3       2012-04-01   15.978388
4       2012-05-01   23.555380
..             ...         ...
139     2023-08-01   94.747376
140     2023-09-01   89.642116
141     2023-10-01    1.523791
142     2023-11-01  109.342708
143     2023-12-01   95.723639

[144 rows x 2 columns]
In [14]:
import plotly.graph_objects as go
import pandas as pd

# Crear la serie de tiempo con Pandas Time Series (ts)
VAFODO_series = pd.Series(Serie['VAFODO'].values, index=Serie['fecha_completa'])

# Crear la figura de Plotly
fig = go.Figure()

# Agregar la serie de tiempo al gráfico
fig.add_trace(go.Scatter(x=VAFODO_series.index, y=VAFODO_series.values, mode='lines'))

# Actualizar el diseño del eje x
fig.update_xaxes(title_text='Fecha', rangeslider_visible=True)

# Actualizar el diseño del eje y
fig.update_yaxes(title_text='Valor de VAFODO')

# Actualizar el título del gráfico
fig.update_layout(title='Serie de tiempo VAFODO')

# Mostrar el gráfico
fig.show()

TRANSFORMACION DE BOX-COX PARA REDUCCION DE LA VARIANZA

In [15]:
import numpy as np
from scipy import stats
from scipy.stats import boxcox
import matplotlib.pyplot as plt
In [16]:
# Calcular el lambda óptimo para la transformación de Box-Cox
fitted_data, fitted_lambda = stats.boxcox(Serie['VAFODO'])
print("Lambda óptimo para Box-Cox:", fitted_lambda)

# Aplicar la transformación de Box-Cox a la serie de tiempo de Apertura
serie_boxcox = boxcox(Serie['VAFODO'], lmbda=fitted_lambda)
print(serie_boxcox)

Lambda óptimo para Box-Cox: 0.40834707479923404
[ 5.10852085  5.49950711  5.82184441  5.14438814  6.44842952  6.4412316
  6.34582517  6.53071295  6.10329901  5.8546702   6.10864499  6.02805918
  5.71646829  5.5638126   5.85791619  6.65395667  6.63815599  6.24445826
  6.37800192  6.74050423  7.33673943  7.34544084  6.82506812  6.24993519
  6.74336775  6.62145455  6.8578093   7.43456938  7.23406832  6.46011486
  7.05232672  7.23189589  7.10665996  7.3047484   7.39371961  7.63645326
  6.06386807  7.22925318  7.06795631  7.34741043  7.34928454  6.83707819
  7.24164381  7.17772518  6.28148124  6.29379786  6.64494545  5.52038758
  5.50730646  5.60673818  6.36419625  6.52656844  6.2377567   6.43884013
  6.24841702  7.25660489  6.74444002  6.66747022  6.86879468  6.36899605
  9.84649471 11.56385693 11.27722176 11.37365638 10.45622964 10.41502527
 11.42312264 11.35079527 10.50665686 10.64446897 10.68308436 10.8247758
  6.50110839  6.68873166  6.62318603 11.18183475 12.16376963 11.87993335
 11.92066752 11.53778177 11.10429974 12.31351759 11.50364885 11.8401968
 11.32794164 11.63995741 12.17273087 12.43452984 12.14473243 11.21805551
 12.54708717 13.1287992  11.52934384 10.90225045 13.2042615  11.19139491
 11.45616211 10.79149589 10.97708045 10.85434894  8.95737564  8.37967215
  9.53060641  9.70909501 10.14973209 10.26773586 11.23084248 10.43560439
 11.12744064 10.95186983 11.63146775 11.80299935 10.477029   12.04955606
 11.69496571 13.1598788  13.18382614 12.97920931 14.32148027 13.68798223
 14.04403793 13.70386105 15.04509417 14.78569033 16.51236865 14.27119717
  8.57356149 14.92617166 14.52088442 13.97886596 14.37931477 13.83580046
 15.45290045 13.38622063 13.58113795 13.01402412 14.24849221 13.09004696
  0.30188257 13.25820185 12.90692786  0.45959336 14.20456032 13.32408987]

ESTIMACION DE LA TENDENCIA USANDO SPLINES

In [17]:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from scipy.interpolate import UnivariateSpline
In [18]:
# Convertir la serie de tiempo a un DataFrame de pandas
df_BoxCox = pd.DataFrame({'Fecha': Serie['fecha_completa'], 'BoxCox': serie_boxcox})
df_BoxCox.set_index('Fecha', inplace=True)  # Establecer la columna 'Fecha' como el índice de tiempo

# Ajuste de splines para estimar la tendencia
dates = df_BoxCox.index.to_julian_date()  # Convertir las fechas a días julianos para facilitar el ajuste spline
values = df_BoxCox['BoxCox'].values

# Ajustar un spline cúbico suave
spline = UnivariateSpline(dates, values, k=1, s = 5)  # k=3 para un spline cúbico
trend_spline = spline(dates)  # Valores estimados de la tendencia

# Graficar la serie de tiempo original y la tendencia estimada por splines
plt.figure(figsize=(12, 8))
plt.plot(df_BoxCox.index, df_BoxCox['BoxCox'], label='Serie de Tiempo Original', color='blue')
plt.plot(df_BoxCox.index, trend_spline, label='Estimación de Tendencia (Splines)', color='red')
plt.xlabel('Fecha')
plt.ylabel('VAFODO')
plt.title('Estimación de Tendencia por Splines')
plt.legend()
plt.grid(True)
plt.show()
No description has been provided for this image
In [19]:
# Serie sin tendencia (remover la tendencia estimada por splines)
sin_tendencia_spline = df_BoxCox['BoxCox'] - trend_spline

INDICE DE INFORMAICON MUTUA (AMI)

In [20]:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import entropy

# Función para calcular la información mutua
def mutual_information(x, y, bins=10):
    hist_xy, _, _ = np.histogram2d(x, y, bins=bins, density=True)
    hist_x, _ = np.histogram(x, bins=bins, density=True)
    hist_y, _ = np.histogram(y, bins=bins, density=True)
    
    entropy_xy = entropy(hist_xy.flatten())
    entropy_x = entropy(hist_x)
    entropy_y = entropy(hist_y)
    
    mi = entropy_x + entropy_y - entropy_xy
    return mi

# Calcular la información mutua para diferentes rezagos
lag_values = np.arange(1, 101)
mutual_info_values = [mutual_information(sin_tendencia_spline[:-lag], sin_tendencia_spline[lag:]) for lag in lag_values]

# Graficar la información mutua en función del rezago
plt.figure(figsize=(10, 6))
plt.plot(lag_values, mutual_info_values, marker='o', linestyle='-')
plt.title('Información Mutua vs Rezago')
plt.xlabel('Rezago')
plt.ylabel('Información Mutua')
plt.grid(True)
plt.show()
No description has been provided for this image

DETECCION DE ESTACIONALIDAD

In [21]:
from scipy.signal import periodogram

# Calcular el periodograma (espectro de potencia) de la serie diferenciada
frequencies, spectrum = periodogram(sin_tendencia_spline)

# Encontrar la frecuencia correspondiente al máximo del espectro
max_frequency = frequencies[np.argmax(spectrum)]
periodo = 1 / max_frequency

# Graficar el periodograma
plt.figure(figsize=(12, 6))
plt.plot(frequencies, spectrum)
plt.axvline(max_frequency, color='red', linestyle='--', label=f'Máximo en {max_frequency:.4f} Hz')
plt.title('Periodograma - Serie sin tendencia vía Kernel')
plt.xlabel('Frecuencia (Hz)')
plt.ylabel('Espectro de Potencia')
plt.legend()
plt.grid(True)
plt.show()

# Calcular el periodo correspondiente a la frecuencia máxima
print(f'Frecuencia dominante: {max_frequency:.2f}')
print(f'Periodo dominante: {periodo:.2f} unidades de tiempo')
No description has been provided for this image 
Frecuencia dominante: 0.34
Periodo dominante: 2.94 unidades de tiempo

REDES NEURONALES APLICADAS A SERIES DE TIEMPO

PREPARACION DE LA SERIE

In [22]:
from __future__ import print_function
import pandas as pd
import numpy as np
%matplotlib inline
from matplotlib import pyplot as plt
import seaborn as sns
import datetime
import scipy as sp
In [23]:
STS = sin_tendencia_spline
print(STS.head())
type(STS)

Fecha
2012-01-01   -0.184492
2012-02-01    0.107021
2012-03-01    0.336302
2012-04-01   -0.440628
2012-05-01    0.352852
Name: BoxCox, dtype: float64
Out[23]:
pandas.core.series.Series
In [24]:
SerieTs = pd.Series(STS, index = Variables['fecha_completa'])
SerieDf = pd.DataFrame(STS, index = Variables['fecha_completa'])
type(SerieTs)
SerieTs.info()
print(SerieTs)
#type(SerieDf)

<class 'pandas.core.series.Series'>
DatetimeIndex: 144 entries, 2012-01-01 to 2023-12-01
Series name: BoxCox
Non-Null Count  Dtype  
--------------  -----  
144 non-null    float64
dtypes: float64(1)
memory usage: 2.2 KB
fecha_completa
2012-01-01   -0.184492
2012-02-01    0.107021
2012-03-01    0.336302
2012-04-01   -0.440628
2012-05-01    0.352852
                ...   
2023-08-01    0.169021
2023-09-01    0.165515
2023-10-01   -0.340459
2023-11-01    0.233495
2023-12-01   -0.062102
Name: BoxCox, Length: 144, dtype: float64
In [30]:
plt.figure(figsize=(5.5, 5.5))
g = sns.boxplot(SerieDf) 
g.set_title('Box Plot de VAFODO')
plt.show()
No description has been provided for this image
In [38]:
import matplotlib.pyplot as plt
from statsmodels.graphics.tsaplots import plot_acf, plot_pacf

# Crear una figura con dos subplots lado a lado
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(15, 5))

# Graficar ACF
plot_acf(SerieDf, lags=100, ax=ax1)
ax1.set_title('Función de Autocorrelación (ACF)')

# Graficar PACF
plot_pacf(SerieDf, lags=40, method='ldbiased', ax=ax2)
ax2.set_title('Función de Autocorrelación Parcial (PACF)')

# Ajustar el espacio entre los subplots
plt.tight_layout()

# Mostrar la figura
plt.show()
No description has been provided for this image
In [39]:
SerieDf.head(5)
Out[39]:
BoxCox
fecha_completa
2012-01-01 -0.184492
2012-02-01 0.107021
2012-03-01 0.336302
2012-04-01 -0.440628
2012-05-01 0.352852
In [42]:
import matplotlib.pyplot as plt
import numpy as np
import scipy.signal as sp

# Supongamos que 'SerieTs' es tu serie de tiempo, asegúrate de tenerla definida previamente.

# Configurar la figura y los subplots
fig, axs = plt.subplots(1, 3, figsize=(18, 6))  # 1 fila, 3 columnas

# Primer subplot: Periodograma
f, Pxx_den = sp.periodogram(sin_tendencia_spline)
max_index_value = np.argmax(Pxx_den, axis=0)
frecuencia_max = f[max_index_value]
print("Frecuencia máxima:", frecuencia_max)
print("Período correspondiente:", 1 / frecuencia_max)
axs[0].plot(f, Pxx_den)
axs[0].set_title('Periodograma')
axs[0].set_xlabel('Frecuencia')
axs[0].set_ylabel('Densidad de potencia')

# Segundo subplot: Welch
f_welch, Pxx_den_welch = sp.welch(sin_tendencia_spline)
axs[1].plot(f_welch, Pxx_den_welch)
axs[1].set_title('Welch')
axs[1].set_xlabel('Frecuencia')
axs[1].set_ylabel('Densidad de potencia')

# Tercer subplot: Espectro de densidad de potencia
axs[2].psd(sin_tendencia_spline)
axs[2].set_title('Espectro de Densidad de Potencia')
axs[2].set_xlabel('Frecuencia')
axs[2].set_ylabel('Densidad de potencia')

# Ajustar automáticamente el espaciado entre subplots
plt.tight_layout()

# Mostrar la figura
plt.show()

Frecuencia máxima: 0.34027777777777773
Período correspondiente: 2.938775510204082
No description has been provided for this image

  1. Periodograma:

    • El eje X representa la frecuencia y el eje Y la densidad de potencia.
    • Los picos en el gráfico indican las frecuencias dominantes en la serie temporal.
    • La frecuencia con la mayor amplitud (el pico más alto) es la frecuencia dominante en la serie.

  2. Método de Welch:

    • Similar al periodograma, pero suele proporcionar una estimación más suave y menos ruidosa del espectro de potencia.
    • Los ejes son los mismos que en el periodograma.
    • Los picos siguen indicando las frecuencias dominantes, pero pueden ser menos pronunciados y más suaves que en el periodograma.

  3. Espectro de Densidad de Potencia (PSD):

    • Este gráfico muestra cómo la potencia de la señal se distribuye en función de la frecuencia.
    • El eje X es la frecuencia y el eje Y es la densidad de potencia en escala logarítmica (dB/Hz).
    • Los picos en este gráfico también indican las frecuencias dominantes en tu serie temporal.

Interpretación general:

  • Busca picos prominentes en los tres gráficos. Si ves picos consistentes en las mismas frecuencias en los tres métodos, es una fuerte indicación de que esas frecuencias son significativas en tu serie temporal.
  • La altura de los picos indica la importancia relativa de cada frecuencia.
  • Las frecuencias bajas (cerca de cero) a menudo representan tendencias a largo plazo, mientras que las frecuencias más altas representan variaciones más rápidas o estacionalidad de corto plazo.
  • Si ves múltiples picos equidistantes, podría indicar la presencia de armónicos, que son múltiplos enteros de una frecuencia fundamental.
  • La ausencia de picos claros y un espectro relativamente plano podría sugerir que la serie temporal se asemeja más a un ruido aleatorio.
In [43]:
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt

# Supongamos que 'SerieTs' es tu serie de tiempo diaria, asegúrate de tenerla definida previamente.

# Aplicar la transformada de Fourier discreta en la serie de tiempo diaria
fft = tf.signal.rfft(sin_tendencia_spline)
frequencies = np.fft.rfftfreq(len(sin_tendencia_spline), d=1)  # Frecuencias correspondientes

# Crear el gráfico
plt.figure(figsize=(12, 6))
plt.plot(frequencies, np.abs(fft))
plt.xlabel('Frecuencia (ciclos por día)')
plt.ylabel('Magnitud de la FFT')
plt.title('Transformada de Fourier de la Serie de Tiempo Diaria')
plt.grid(True)
plt.show()
No description has been provided for this image
In [44]:
# Obtener el índice de la serie de tiempo como un objeto DatetimeIndex
date_time_index = STS.index
date_time_index = pd.Series(date_time_index)
type(date_time_index)

# Convertir cada valor de fecha y hora a un timestamp en segundos
timestamp_s = date_time_index.apply(pd.Timestamp.timestamp)

# Mostrar el resultado (opcional)
print(timestamp_s)

0      1.325376e+09
1      1.328054e+09
2      1.330560e+09
3      1.333238e+09
4      1.335830e+09
           ...     
139    1.690848e+09
140    1.693526e+09
141    1.696118e+09
142    1.698797e+09
143    1.701389e+09
Name: Fecha, Length: 144, dtype: float64
In [49]:
df = pd.DataFrame()
day = 24*60*60
year = (365.2425)*day

df['VAFODO'] = STS.values
df['Year sin'] = np.sin(timestamp_s * (2 * np.pi / year))
df['Year cos'] = np.cos(timestamp_s * (2 * np.pi / year))
In [50]:
plt.plot(np.array(df['Year sin'])[:365])
plt.plot(np.array(df['Year cos'])[:365])
plt.xlabel('Time [D]')
plt.title('Time of year signal')
plt.show()
No description has been provided for this image
In [52]:
df['Fecha'] = STS.index
df.columns
df.head(5)
Out[52]:
VAFODO Year sin Year cos Fecha
0 -0.184492 -0.003183 0.999995 2012-01-01
1 0.107021 0.505623 0.862755 2012-02-01
2 0.336302 0.856776 0.515689 2012-03-01
3 -0.440628 0.999964 0.008526 2012-04-01
4 0.352852 0.873934 -0.486044 2012-05-01
In [53]:
df1 = df.set_index('Fecha')
df1
Out[53]:
VAFODO Year sin Year cos
Fecha
2012-01-01 -0.184492 -0.003183 0.999995
2012-02-01 0.107021 0.505623 0.862755
2012-03-01 0.336302 0.856776 0.515689
2012-04-01 -0.440628 0.999964 0.008526
2012-05-01 0.352852 0.873934 -0.486044
... ... ... ...
2023-08-01 0.169021 -0.486373 -0.873751
2023-09-01 0.165515 -0.863021 -0.505168
2023-10-01 -0.340459 -0.999909 -0.013493
2023-11-01 0.233495 -0.867922 0.496700
2023-12-01 -0.062102 -0.509773 0.860309

144 rows × 3 columns

DIVISION ENTRENAMIENTO, VALIDACION Y PRUEBA

In [54]:
column_indices = {name: i for i, name in enumerate(df1.columns)}

n = len(df1)
train_df1 = df1[0:int(n*0.7)]
val_df1 = df1[int(n*0.7):int(n*0.9)]
test_df1 = df1[int(n*0.9):]

num_features = df1.shape[1]
num_features
Out[54]:
3
In [55]:
train_df1
Out[55]:
VAFODO Year sin Year cos
Fecha
2012-01-01 -0.184492 -0.003183 0.999995
2012-02-01 0.107021 0.505623 0.862755
2012-03-01 0.336302 0.856776 0.515689
2012-04-01 -0.440628 0.999964 0.008526
2012-05-01 0.352852 0.873934 -0.486044
... ... ... ...
2019-12-01 -0.117771 -0.510216 0.860046
2020-01-01 0.274925 -0.002150 0.999998
2020-02-01 -0.261813 0.506513 0.862232
2020-03-01 0.043447 0.857307 0.514805
2020-04-01 0.048644 0.999972 0.007494

100 rows × 3 columns

In [56]:
val_df1
Out[56]:
VAFODO Year sin Year cos
Fecha
2020-05-01 -0.012226 0.873432 -0.486946
2020-06-01 -0.013323 0.504602 -0.863352
2020-07-01 0.039410 0.012837 -0.999918
2020-08-01 -0.051023 -0.497269 -0.867596
2020-09-01 0.120693 -0.869275 -0.494328
2020-10-01 -0.194687 -1.000000 -0.000978
2020-11-01 0.320590 -0.861638 0.507523
2020-12-01 -0.435749 -0.498966 0.866622
2021-01-01 0.296284 0.010881 0.999941
2021-02-01 -0.216234 0.517705 0.855559
2021-03-01 0.159025 0.855152 0.518377
2021-04-01 -0.006391 0.999932 0.011665
2021-05-01 -0.040475 0.875456 -0.483298
2021-06-01 0.036150 0.508199 -0.861240
2021-07-01 -0.030250 0.017008 -0.999855
2021-08-01 -0.017308 -0.493646 -0.869663
2021-09-01 0.077064 -0.867206 -0.497950
2021-10-01 -0.059400 -0.999987 -0.005150
2021-11-01 0.026500 -0.863748 0.503924
2021-12-01 -0.019317 -0.502577 0.864532
2022-01-01 0.016923 0.006709 0.999977
2022-02-01 -0.026250 0.514132 0.857711
2022-03-01 0.031352 0.852982 0.521940
2022-04-01 -0.041823 0.999875 0.015837
2022-05-01 0.029148 0.877465 -0.479641
2022-06-01 0.065711 0.511788 -0.859112
2022-07-01 -0.147462 0.021179 -0.999776
2022-08-01 0.106265 -0.490013 -0.871715
2022-09-01 -0.024756 -0.865121 -0.501563
In [57]:
test_df1
Out[57]:
VAFODO Year sin Year cos
Fecha
2022-10-01 -0.151257 -0.999957 -0.009322
2022-11-01 0.307616 -0.865843 0.500317
2022-12-01 -0.179360 -0.506179 0.862428
2023-01-01 0.042220 0.002537 0.999997
2023-02-01 -0.145414 0.510549 0.859848
2023-03-01 0.234330 0.850798 0.525494
2023-04-01 -0.128154 0.999800 0.020008
2023-05-01 -0.020475 0.879458 -0.475977
2023-06-01 0.202248 0.515367 -0.856970
2023-07-01 -0.329596 0.025350 -0.999679
2023-08-01 0.169021 -0.486373 -0.873751
2023-09-01 0.165515 -0.863021 -0.505168
2023-10-01 -0.340459 -0.999909 -0.013493
2023-11-01 0.233495 -0.867922 0.496700
2023-12-01 -0.062102 -0.509773 0.860309
In [58]:
#Normalizacion de las variables 

train_mean = train_df1.mean()
train_std = train_df1.std()

train_df = (train_df1 - train_mean) / train_std
val_df = (val_df1 - train_mean) / train_std
test_df = (test_df1 - train_mean) / train_std
In [60]:
#Creacion de las variables rezagadas

# Crear un DataFrame vacío para almacenar los rezagos
df2 = pd.DataFrame()

# Generar los primeros 7 rezagos consecutivos
num_lags_consecutivos = 3
for i in range(num_lags_consecutivos, 0, -1):
    df2[f't-{i}'] = STS.shift(i)

# Generar rezagos que son múltiplos de 7 hasta 70
num_lags_multiples_7 = 24
for i in range(3, num_lags_multiples_7 + 1, 3):
    df2[f't-{i}'] = STS.shift(i)

# Mostrar el DataFrame resultante
print(df2)

                 t-3       t-2       t-1       t-6       t-9      t-12  \
Fecha                                                                    
2012-01-01       NaN       NaN       NaN       NaN       NaN       NaN   
2012-02-01       NaN       NaN -0.184492       NaN       NaN       NaN   
2012-03-01       NaN -0.184492  0.107021       NaN       NaN       NaN   
2012-04-01 -0.184492  0.107021  0.336302       NaN       NaN       NaN   
2012-05-01  0.107021  0.336302 -0.440628       NaN       NaN       NaN   
...              ...       ...       ...       ...       ...       ...   
2023-08-01 -0.020475  0.202248 -0.329596 -0.145414  0.307616  0.106265   
2023-09-01  0.202248 -0.329596  0.169021  0.234330 -0.179360 -0.024756   
2023-10-01 -0.329596  0.169021  0.165515 -0.128154  0.042220 -0.151257   
2023-11-01  0.169021  0.165515 -0.340459 -0.020475 -0.145414  0.307616   
2023-12-01  0.165515 -0.340459  0.233495  0.202248  0.234330 -0.179360   

                t-15      t-18      t-21      t-24  
Fecha                                               
2012-01-01       NaN       NaN       NaN       NaN  
2012-02-01       NaN       NaN       NaN       NaN  
2012-03-01       NaN       NaN       NaN       NaN  
2012-04-01       NaN       NaN       NaN       NaN  
2012-05-01       NaN       NaN       NaN       NaN  
...              ...       ...       ...       ...  
2023-08-01  0.029148 -0.026250  0.026500 -0.017308  
2023-09-01  0.065711  0.031352 -0.019317  0.077064  
2023-10-01 -0.147462 -0.041823  0.016923 -0.059400  
2023-11-01  0.106265  0.029148 -0.026250  0.026500  
2023-12-01 -0.024756  0.065711  0.031352 -0.019317  

[144 rows x 10 columns]
In [61]:
df2['t'] = df1['VAFODO']
print(df2.head(8))

                 t-3       t-2       t-1       t-6  t-9  t-12  t-15  t-18  \
Fecha                                                                       
2012-01-01       NaN       NaN       NaN       NaN  NaN   NaN   NaN   NaN   
2012-02-01       NaN       NaN -0.184492       NaN  NaN   NaN   NaN   NaN   
2012-03-01       NaN -0.184492  0.107021       NaN  NaN   NaN   NaN   NaN   
2012-04-01 -0.184492  0.107021  0.336302       NaN  NaN   NaN   NaN   NaN   
2012-05-01  0.107021  0.336302 -0.440628       NaN  NaN   NaN   NaN   NaN   
2012-06-01  0.336302 -0.440628  0.352852       NaN  NaN   NaN   NaN   NaN   
2012-07-01 -0.440628  0.352852 -0.181926 -0.184492  NaN   NaN   NaN   NaN   
2012-08-01  0.352852 -0.181926 -0.120903  0.107021  NaN   NaN   NaN   NaN   

            t-21  t-24         t  
Fecha                             
2012-01-01   NaN   NaN -0.184492  
2012-02-01   NaN   NaN  0.107021  
2012-03-01   NaN   NaN  0.336302  
2012-04-01   NaN   NaN -0.440628  
2012-05-01   NaN   NaN  0.352852  
2012-06-01   NaN   NaN -0.181926  
2012-07-01   NaN   NaN -0.120903  
2012-08-01   NaN   NaN  0.225629
In [62]:
# Create a new subsetted dataframe, removing Nans from first 7 rows
df3 = df2.dropna()
print(df3.head(7))
df3.shape

                 t-3       t-2       t-1       t-6       t-9      t-12  \
Fecha                                                                    
2014-01-01 -0.133034 -0.047328 -0.035934  0.043254  0.392747 -0.161444   
2014-02-01 -0.047328 -0.035934  0.215537  0.020370 -0.061695 -0.277339   
2014-03-01 -0.035934  0.215537 -0.148338  0.231219 -0.269849  0.049968   
2014-04-01  0.215537 -0.148338 -0.130530 -0.133034  0.043254  0.392747   
2014-05-01 -0.148338 -0.130530  0.204268 -0.047328  0.020370 -0.061695   
2014-06-01 -0.130530  0.204268  0.128602 -0.035934  0.231219 -0.269849   
2014-07-01  0.204268  0.128602 -0.516356  0.215537 -0.133034  0.043254   

                t-15      t-18      t-21      t-24         t  
Fecha                                                         
2014-01-01 -0.132340 -0.120903 -0.440628 -0.184492  0.215537  
2014-02-01  0.158396  0.225629  0.352852  0.107021 -0.148338  
2014-03-01  0.113385 -0.040141 -0.181926  0.336302 -0.130530  
2014-04-01 -0.161444 -0.132340 -0.120903 -0.440628  0.204268  
2014-05-01 -0.277339  0.158396  0.225629  0.352852  0.128602  
2014-06-01  0.049968  0.113385 -0.040141 -0.181926 -0.516356  
2014-07-01  0.392747 -0.161444 -0.132340 -0.120903  0.200690
Out[62]:
(120, 11)
In [63]:
#DIVIDIR DATOS
DFsplit = df3.values

X1= DFsplit[:, 0:-1]  
y1 =DFsplit[:,-1]
In [65]:
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler, MinMaxScaler
In [66]:
X_train_full, X_test, y_train_full, y_test=train_test_split(X1,y1,test_size=0.1, train_size=0.9,shuffle=False)
In [67]:
print(X_train_full.shape)
print(X_test.shape)

print(y_train_full.shape)
print(y_test.shape)

(108, 10)
(12, 10)
(108,)
(12,)
In [68]:
X_train, X_val, y_train, y_val=train_test_split(X_train_full,y_train_full,test_size=0.2, train_size=0.8,shuffle=False)
In [69]:
print(X_train.shape)
print(X_val.shape)

print(y_train.shape)
print(y_val.shape)

(86, 10)
(22, 10)
(86,)
(22,)

NORMALIZACION DE LA COVARIABLES - UTILIZANDO MIN MAX SCALES¶

In [70]:
scaler=MinMaxScaler()
X_train=scaler.fit_transform(X_train)
X_val=scaler.transform(X_val)
X_test=scaler.transform(X_test)

REDES NEURONALES DE UNA CAPA

In [71]:
import tensorflow.keras as keras
In [72]:
model=keras.models.Sequential([keras.layers.Dense(64,activation="relu",input_shape=X_train.shape[1:]),keras.layers.Dense(1)])
model.summary()

Model: "sequential"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ Layer (type)                         ┃ Output Shape                ┃         Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ dense (Dense)                        │ (None, 64)                  │             704 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense_1 (Dense)                      │ (None, 1)                   │              65 │
└──────────────────────────────────────┴─────────────────────────────┴─────────────────┘

 Total params: 769 (3.00 KB)

 Trainable params: 769 (3.00 KB)

 Non-trainable params: 0 (0.00 B)
In [73]:
model.compile(loss="mean_squared_error",optimizer="adam")
history=model.fit(X_train,y_train,epochs=50,validation_data=(X_val,y_val))

Epoch 1/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 1s 91ms/step - loss: 0.0540 - val_loss: 0.0204
Epoch 2/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 19ms/step - loss: 0.0389 - val_loss: 0.0262
Epoch 3/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 18ms/step - loss: 0.0474 - val_loss: 0.0229
Epoch 4/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 21ms/step - loss: 0.0419 - val_loss: 0.0188
Epoch 5/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 25ms/step - loss: 0.0370 - val_loss: 0.0177
Epoch 6/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 25ms/step - loss: 0.0380 - val_loss: 0.0171
Epoch 7/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 22ms/step - loss: 0.0389 - val_loss: 0.0161
Epoch 8/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 23ms/step - loss: 0.0316 - val_loss: 0.0167
Epoch 9/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 24ms/step - loss: 0.0363 - val_loss: 0.0172
Epoch 10/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 22ms/step - loss: 0.0358 - val_loss: 0.0158
Epoch 11/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 21ms/step - loss: 0.0350 - val_loss: 0.0144
Epoch 12/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 74ms/step - loss: 0.0293 - val_loss: 0.0135
Epoch 13/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 23ms/step - loss: 0.0314 - val_loss: 0.0129
Epoch 14/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 23ms/step - loss: 0.0315 - val_loss: 0.0124
Epoch 15/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 22ms/step - loss: 0.0341 - val_loss: 0.0124
Epoch 16/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 26ms/step - loss: 0.0275 - val_loss: 0.0130
Epoch 17/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 26ms/step - loss: 0.0285 - val_loss: 0.0123
Epoch 18/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 32ms/step - loss: 0.0257 - val_loss: 0.0109
Epoch 19/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 29ms/step - loss: 0.0254 - val_loss: 0.0105
Epoch 20/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 31ms/step - loss: 0.0303 - val_loss: 0.0107
Epoch 21/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 31ms/step - loss: 0.0253 - val_loss: 0.0109
Epoch 22/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 34ms/step - loss: 0.0285 - val_loss: 0.0106
Epoch 23/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 33ms/step - loss: 0.0281 - val_loss: 0.0097
Epoch 24/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 27ms/step - loss: 0.0280 - val_loss: 0.0092
Epoch 25/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 33ms/step - loss: 0.0266 - val_loss: 0.0090
Epoch 26/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 27ms/step - loss: 0.0270 - val_loss: 0.0097
Epoch 27/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 30ms/step - loss: 0.0271 - val_loss: 0.0096
Epoch 28/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 29ms/step - loss: 0.0292 - val_loss: 0.0097
Epoch 29/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 29ms/step - loss: 0.0238 - val_loss: 0.0088
Epoch 30/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 35ms/step - loss: 0.0249 - val_loss: 0.0084
Epoch 31/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 36ms/step - loss: 0.0269 - val_loss: 0.0086
Epoch 32/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 32ms/step - loss: 0.0261 - val_loss: 0.0094
Epoch 33/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 34ms/step - loss: 0.0233 - val_loss: 0.0091
Epoch 34/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 38ms/step - loss: 0.0237 - val_loss: 0.0084
Epoch 35/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 32ms/step - loss: 0.0218 - val_loss: 0.0080
Epoch 36/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 35ms/step - loss: 0.0234 - val_loss: 0.0079
Epoch 37/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 35ms/step - loss: 0.0237 - val_loss: 0.0086
Epoch 38/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 33ms/step - loss: 0.0215 - val_loss: 0.0097
Epoch 39/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 28ms/step - loss: 0.0223 - val_loss: 0.0088
Epoch 40/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 38ms/step - loss: 0.0206 - val_loss: 0.0079
Epoch 41/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 26ms/step - loss: 0.0208 - val_loss: 0.0077
Epoch 42/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 29ms/step - loss: 0.0231 - val_loss: 0.0085
Epoch 43/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 33ms/step - loss: 0.0204 - val_loss: 0.0097
Epoch 44/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 33ms/step - loss: 0.0231 - val_loss: 0.0090
Epoch 45/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 29ms/step - loss: 0.0212 - val_loss: 0.0076
Epoch 46/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 34ms/step - loss: 0.0198 - val_loss: 0.0076
Epoch 47/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 28ms/step - loss: 0.0212 - val_loss: 0.0084
Epoch 48/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 32ms/step - loss: 0.0214 - val_loss: 0.0093
Epoch 49/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 36ms/step - loss: 0.0185 - val_loss: 0.0082
Epoch 50/50
3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 22ms/step - loss: 0.0211 - val_loss: 0.0075
In [74]:
import matplotlib.pyplot as plt

# Crear un DataFrame a partir del historial
df_history = pd.DataFrame(history.history)

# Verificar las métricas disponibles en el historial
print(df_history.keys())

# Graficar las métricas de pérdida (loss) y precisión (accuracy), si están disponibles
if 'loss' in df_history:
    plt.figure(figsize=(10, 6))
    plt.plot(df_history['loss'], label='Pérdida (Entrenamiento)', color='blue', linestyle='-')
    if 'val_loss' in df_history:
        plt.plot(df_history['val_loss'], label='Pérdida (Validación)', color='orange', linestyle='--')
    plt.title('Gráfico de Historial de Pérdida')
    plt.xlabel('Época')
    plt.ylabel('Pérdida')
    plt.legend()
    plt.grid(True)
    plt.show()
else:
    print("No se encontraron métricas de pérdida (loss) en el historial.")

Index(['loss', 'val_loss'], dtype='object')
No description has been provided for this image

A pesar de que en ambas curvas podemos observar un tendencia a la disminucion de la perdidad tambien podemos observar que la curva de perdidad en el conjunto de validacion es inusualmente oscilante, lo cual no sabemos que puede significar Preguntar al profe

In [75]:
mse_test=model.evaluate(X_test,y_test)
print(mse_test)

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 152ms/step - loss: 0.0174
0.01735829748213291
In [76]:
ypred=model.predict(X_test)

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 111ms/step
In [87]:
import matplotlib.pyplot as plt

# Definir el tamaño y los subplots de la figura
fig, ax = plt.subplots(figsize=(10, 6))

# Graficar valores reales (y_test) y valores predichos (ypred)
ax.plot(range(y_test.size), y_test, linestyle='-', marker='*', color='blue', label='Actual')
ax.plot(range(ypred.size), ypred, linestyle='-', marker='.', color='orange', label='Predicted')

# Agregar título y etiquetas de ejes
ax.set_title('Actual vs Predicted OPEN')
ax.set_xlabel('Index')
ax.set_ylabel('OPEN')

# Agregar leyenda
ax.legend()

# Mostrar la gráfica mejorada
plt.show()
No description has been provided for this image

REDES NEURONALES MULTICAPA

In [88]:
input_layer = keras.layers.Input(shape=X_train.shape[1:])
hidden1 = keras.layers.Dense(32,activation="linear")(input_layer)
hidden2 = keras.layers.Dense(16,activation="linear")(hidden1)
hidden3 = keras.layers.Dense(16,activation="linear")(hidden2)
output_layer = keras.layers.Dense(1, activation='linear')(hidden3)
In [89]:
ts_model=keras.Model(inputs=[input_layer],outputs=[output_layer])
ts_model.summary()

Model: "functional_1"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ Layer (type)                         ┃ Output Shape                ┃         Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ input_layer_1 (InputLayer)           │ (None, 10)                  │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense_2 (Dense)                      │ (None, 32)                  │             352 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense_3 (Dense)                      │ (None, 16)                  │             528 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense_4 (Dense)                      │ (None, 16)                  │             272 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense_5 (Dense)                      │ (None, 1)                   │              17 │
└──────────────────────────────────────┴─────────────────────────────┴─────────────────┘

 Total params: 1,169 (4.57 KB)

 Trainable params: 1,169 (4.57 KB)

 Non-trainable params: 0 (0.00 B)
In [90]:
print("Input shape:", X_train.shape)
print("Target shape:", y_train.shape)

Input shape: (86, 10)
Target shape: (86,)
In [91]:
ts_model.compile(loss="mean_squared_error",optimizer="adam")
history=ts_model.fit(X_train,y_train,epochs=50,batch_size=16,validation_data=(X_val,y_val))

Epoch 1/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 2s 42ms/step - loss: 0.4079 - val_loss: 0.0423
Epoch 2/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step - loss: 0.0801 - val_loss: 0.0782
Epoch 3/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 46ms/step - loss: 0.1153 - val_loss: 0.0538
Epoch 4/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step - loss: 0.0680 - val_loss: 0.0254
Epoch 5/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step - loss: 0.0605 - val_loss: 0.0315
Epoch 6/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step - loss: 0.0590 - val_loss: 0.0265
Epoch 7/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step - loss: 0.0464 - val_loss: 0.0206
Epoch 8/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step - loss: 0.0380 - val_loss: 0.0226
Epoch 9/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step - loss: 0.0499 - val_loss: 0.0196
Epoch 10/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step - loss: 0.0383 - val_loss: 0.0168
Epoch 11/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step - loss: 0.0455 - val_loss: 0.0156
Epoch 12/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step - loss: 0.0413 - val_loss: 0.0148
Epoch 13/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 15ms/step - loss: 0.0390 - val_loss: 0.0139
Epoch 14/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 24ms/step - loss: 0.0294 - val_loss: 0.0131
Epoch 15/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 20ms/step - loss: 0.0340 - val_loss: 0.0135
Epoch 16/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 15ms/step - loss: 0.0303 - val_loss: 0.0125
Epoch 17/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 18ms/step - loss: 0.0321 - val_loss: 0.0114
Epoch 18/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 16ms/step - loss: 0.0344 - val_loss: 0.0110
Epoch 19/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 14ms/step - loss: 0.0349 - val_loss: 0.0106
Epoch 20/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 16ms/step - loss: 0.0253 - val_loss: 0.0107
Epoch 21/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 16ms/step - loss: 0.0250 - val_loss: 0.0106
Epoch 22/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 14ms/step - loss: 0.0312 - val_loss: 0.0103
Epoch 23/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 14ms/step - loss: 0.0279 - val_loss: 0.0097
Epoch 24/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step - loss: 0.0275 - val_loss: 0.0096
Epoch 25/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 13ms/step - loss: 0.0242 - val_loss: 0.0103
Epoch 26/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step - loss: 0.0296 - val_loss: 0.0097
Epoch 27/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step - loss: 0.0250 - val_loss: 0.0096
Epoch 28/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 13ms/step - loss: 0.0284 - val_loss: 0.0096
Epoch 29/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 14ms/step - loss: 0.0244 - val_loss: 0.0096
Epoch 30/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step - loss: 0.0267 - val_loss: 0.0094
Epoch 31/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 20ms/step - loss: 0.0271 - val_loss: 0.0096
Epoch 32/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 15ms/step - loss: 0.0256 - val_loss: 0.0092
Epoch 33/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 14ms/step - loss: 0.0236 - val_loss: 0.0095
Epoch 34/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step - loss: 0.0244 - val_loss: 0.0097
Epoch 35/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 13ms/step - loss: 0.0275 - val_loss: 0.0091
Epoch 36/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step - loss: 0.0188 - val_loss: 0.0091
Epoch 37/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 13ms/step - loss: 0.0242 - val_loss: 0.0096
Epoch 38/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 14ms/step - loss: 0.0222 - val_loss: 0.0092
Epoch 39/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 13ms/step - loss: 0.0217 - val_loss: 0.0094
Epoch 40/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step - loss: 0.0233 - val_loss: 0.0098
Epoch 41/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 13ms/step - loss: 0.0263 - val_loss: 0.0096
Epoch 42/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step - loss: 0.0265 - val_loss: 0.0096
Epoch 43/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step - loss: 0.0213 - val_loss: 0.0100
Epoch 44/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 13ms/step - loss: 0.0290 - val_loss: 0.0099
Epoch 45/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step - loss: 0.0279 - val_loss: 0.0097
Epoch 46/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step - loss: 0.0209 - val_loss: 0.0096
Epoch 47/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 13ms/step - loss: 0.0246 - val_loss: 0.0103
Epoch 48/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step - loss: 0.0213 - val_loss: 0.0089
Epoch 49/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step - loss: 0.0244 - val_loss: 0.0087
Epoch 50/50
6/6 ━━━━━━━━━━━━━━━━━━━━ 0s 13ms/step - loss: 0.0231 - val_loss: 0.0088
In [92]:
import matplotlib.pyplot as plt

# Crear un DataFrame a partir del historial
df_history = pd.DataFrame(history.history)

# Verificar las métricas disponibles en el historial
print(df_history.keys())

# Graficar las métricas de pérdida (loss) y precisión (accuracy), si están disponibles
if 'loss' in df_history:
    plt.figure(figsize=(10, 6))
    plt.plot(df_history['loss'], label='Pérdida (Entrenamiento)', color='blue', linestyle='-')
    if 'val_loss' in df_history:
        plt.plot(df_history['val_loss'], label='Pérdida (Validación)', color='orange', linestyle='--')
    plt.title('Gráfico de Historial de Pérdida')
    plt.xlabel('Época')
    plt.ylabel('Pérdida')
    plt.legend()
    plt.grid(True)
    plt.show()
else:
    print("No se encontraron métricas de pérdida (loss) en el historial.")

Index(['loss', 'val_loss'], dtype='object')
No description has been provided for this image

Usando Redes Neuronales Multicapa se puede ver como ambas curvas presentan un comportamiento similar a partir de pocas epocas, y ademas de presentar la misma tendencia, ninguna de las curvar presenta un comportamiento oscilante.

In [93]:
mse_test=ts_model.evaluate(X_test,y_test)
print(mse_test)

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 39ms/step - loss: 0.0165
0.01654651388525963
In [94]:
preds_ts_model=ts_model.predict(X_test)

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 76ms/step
In [ ]:
## HACER LA GRAFICA DEL MODELO PARA LOS DATOS COMPLETOS
In [95]:
import matplotlib.pyplot as plt

# Definir el tamaño y los subplots de la figura
fig, ax = plt.subplots(figsize=(10, 6))

# Graficar valores reales (y_test) y valores predichos (preds_ts_model)
ax.plot(range(y_test.size), y_test, linestyle='-', marker='*', color='blue', label='Actual')
ax.plot(range(preds_ts_model.size), preds_ts_model, linestyle='-', marker='.', color='orange', label='Predicted')

# Agregar título y etiquetas de ejes
ax.set_title('Actual vs Predicted Apertura Transformada')
ax.set_xlabel('Index')
ax.set_ylabel('Apertura Transformada')

# Agregar leyenda
ax.legend()

# Mostrar la gráfica mejorada
plt.show()
No description has been provided for this image

BUSQUEDA DE HIPERPARAMETROS

In [96]:
from tensorflow import keras
import keras_tuner as kt
from tensorflow.keras import layers
In [97]:
def build_model(hp):
    model = keras.Sequential()
    model.add(layers.Dense(units=hp.Int("num_units", min_value=32, max_value=64, step=32),
                activation=hp.Choice("activation", ["relu", "tanh"])))
    # Tune the number of layers.
    for i in range(hp.Int("num_layers", 1, 2)):
        model.add(
            layers.Dense(
                # Tune number of units separately.
                units=hp.Int(f"units_{i}", min_value=32, max_value=64, step=32),
                activation=hp.Choice("activation", ["relu", "tanh"]),
            )
        )
    if hp.Boolean("dropout"):
        model.add(layers.Dropout(rate=0.25))
    model.add(layers.Dense(1, activation="linear"))
    learning_rate = hp.Float("lr", min_value=1e-3, max_value=1e-2,step=0.003)
    model.compile(
        optimizer=keras.optimizers.Adam(learning_rate=learning_rate),
        loss="mean_squared_error",
        metrics=["mean_squared_error"]
    )
    return model


build_model(kt.HyperParameters())
Out[97]:
<Sequential name=sequential_1, built=False>
In [98]:
tuner_GridSearch_mlp = kt.GridSearch(
    hypermodel=build_model,
    objective="val_loss",
    max_trials=None,
    executions_per_trial=1,
    overwrite=True,
    directory="dirsalida",
    project_name="helloworld",
)
In [99]:
tuner_GridSearch_mlp.search_space_summary()

Search space summary
Default search space size: 6
num_units (Int)
{'default': None, 'conditions': [], 'min_value': 32, 'max_value': 64, 'step': 32, 'sampling': 'linear'}
activation (Choice)
{'default': 'relu', 'conditions': [], 'values': ['relu', 'tanh'], 'ordered': False}
num_layers (Int)
{'default': None, 'conditions': [], 'min_value': 1, 'max_value': 2, 'step': 1, 'sampling': 'linear'}
units_0 (Int)
{'default': None, 'conditions': [], 'min_value': 32, 'max_value': 64, 'step': 32, 'sampling': 'linear'}
dropout (Boolean)
{'default': False, 'conditions': []}
lr (Float)
{'default': 0.001, 'conditions': [], 'min_value': 0.001, 'max_value': 0.01, 'step': 0.003, 'sampling': 'linear'}
In [100]:
stop_early=tf.keras.callbacks.EarlyStopping(monitor="val_loss",patience=2)
In [101]:
tuner_GridSearch_mlp.search(X_train,y_train, epochs=20, validation_data=(X_val,y_val),callbacks=[stop_early])

Trial 240 Complete [00h 00m 06s]
val_loss: 0.012469977140426636

Best val_loss So Far: 0.006244177930057049
Total elapsed time: 00h 19m 18s
In [102]:
tuner_GridSearch_mlp.results_summary()

Results summary
Results in dirsalida\helloworld
Showing 10 best trials
Objective(name="val_loss", direction="min")

Trial 0060 summary
Hyperparameters:
num_units: 32
activation: tanh
num_layers: 1
units_0: 32
dropout: True
lr: 0.007
units_1: 32
Score: 0.006244177930057049

Trial 0032 summary
Hyperparameters:
num_units: 32
activation: relu
num_layers: 2
units_0: 64
dropout: False
lr: 0.001
units_1: 32
Score: 0.006260564550757408

Trial 0192 summary
Hyperparameters:
num_units: 64
activation: tanh
num_layers: 1
units_0: 64
dropout: False
lr: 0.001
units_1: 32
Score: 0.006301934830844402

Trial 0217 summary
Hyperparameters:
num_units: 64
activation: tanh
num_layers: 2
units_0: 32
dropout: True
lr: 0.001
units_1: 64
Score: 0.0064923884347081184

Trial 0124 summary
Hyperparameters:
num_units: 64
activation: relu
num_layers: 1
units_0: 32
dropout: True
lr: 0.007
units_1: 32
Score: 0.006633879151195288

Trial 0094 summary
Hyperparameters:
num_units: 32
activation: tanh
num_layers: 2
units_0: 32
dropout: True
lr: 0.010000000000000002
units_1: 32
Score: 0.006981552112847567

Trial 0069 summary
Hyperparameters:
num_units: 32
activation: tanh
num_layers: 1
units_0: 64
dropout: False
lr: 0.007
units_1: 64
Score: 0.007090679835528135

Trial 0015 summary
Hyperparameters:
num_units: 32
activation: relu
num_layers: 1
units_0: 64
dropout: True
lr: 0.010000000000000002
Score: 0.007264276035130024

Trial 0113 summary
Hyperparameters:
num_units: 64
activation: relu
num_layers: 1
units_0: 32
dropout: False
lr: 0.001
units_1: 64
Score: 0.007284015417098999

Trial 0132 summary
Hyperparameters:
num_units: 64
activation: relu
num_layers: 1
units_0: 64
dropout: False
lr: 0.007
units_1: 32
Score: 0.007293394301086664

AJUSTE DEL MEJOR MODELO

In [103]:
x_all = np.concatenate((X_train, X_val))
y_all = np.concatenate((y_train, y_val))
In [104]:
callback=tf.keras.callbacks.EarlyStopping(monitor="loss",patience=5)
best_hps_mlp = tuner_GridSearch_mlp.get_best_hyperparameters(2)
model_mlp = build_model(best_hps_mlp[0])
model_mlp.fit(x_all,y_all, epochs=50,callbacks=[callback])

Epoch 1/50
4/4 ━━━━━━━━━━━━━━━━━━━━ 3s 17ms/step - loss: 0.1002 - mean_squared_error: 0.1002
Epoch 2/50
4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step - loss: 0.0792 - mean_squared_error: 0.0792 
Epoch 3/50
4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step - loss: 0.0664 - mean_squared_error: 0.0664 
Epoch 4/50
4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - loss: 0.0464 - mean_squared_error: 0.0464 
Epoch 5/50
4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 4ms/step - loss: 0.0368 - mean_squared_error: 0.0368 
Epoch 6/50
4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 4ms/step - loss: 0.0352 - mean_squared_error: 0.0352 
Epoch 7/50
4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 0.0274 - mean_squared_error: 0.0274 
Epoch 8/50
4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 4ms/step - loss: 0.0363 - mean_squared_error: 0.0363 
Epoch 9/50
4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 0.0293 - mean_squared_error: 0.0293 
Epoch 10/50
4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 0.0269 - mean_squared_error: 0.0269 
Epoch 11/50
4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 0.0313 - mean_squared_error: 0.0313 
Epoch 12/50
4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 4ms/step - loss: 0.0229 - mean_squared_error: 0.0229  
Epoch 13/50
4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 0.0233 - mean_squared_error: 0.0233 
Epoch 14/50
4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 4ms/step - loss: 0.0210 - mean_squared_error: 0.0210 
Epoch 15/50
4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 0.0257 - mean_squared_error: 0.0257 
Epoch 16/50
4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 0.0247 - mean_squared_error: 0.0247 
Epoch 17/50
4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 0.0283 - mean_squared_error: 0.0283 
Epoch 18/50
4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 0.0239 - mean_squared_error: 0.0239 
Epoch 19/50
4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 0.0224 - mean_squared_error: 0.0224 
Epoch 20/50
4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 0.0218 - mean_squared_error: 0.0218 
Epoch 21/50
4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 0.0235 - mean_squared_error: 0.0235
Out[104]:
<keras.src.callbacks.history.History at 0x12ea9716fc0>
In [105]:
model_mlp.layers[0].get_weights()[0]
Out[105]:
array([[-0.15095809,  0.20001666,  0.3007578 , -0.13121086, -0.31207412,
         0.00185723,  0.24665833,  0.22933015,  0.11014105, -0.07309569,
         0.0926317 ,  0.17205709,  0.20608707,  0.27308628,  0.06966268,
         0.23907435,  0.05753055,  0.21425387, -0.13736536,  0.08113011,
         0.24833979,  0.30252326,  0.0612711 , -0.3364436 ,  0.08647759,
         0.18333593,  0.29347402, -0.19357349, -0.22845136, -0.14580955,
        -0.03726181,  0.04177629],
       [ 0.06818902,  0.24597141, -0.09713513,  0.3030726 ,  0.05974922,
        -0.34589663, -0.33459726, -0.2875955 ,  0.05262516,  0.28374103,
        -0.1753265 ,  0.21589702, -0.27739587, -0.08702733, -0.3447588 ,
        -0.05863032, -0.10086022,  0.08792235, -0.26207972, -0.08014409,
        -0.35585025, -0.1870218 , -0.12146159,  0.10534648, -0.39503014,
         0.10674063, -0.05817251,  0.18091859,  0.1781484 , -0.17314738,
        -0.1122386 , -0.264332  ],
       [ 0.08480758,  0.06777472,  0.23133221,  0.20151792, -0.35345423,
         0.34853584,  0.15049875,  0.29083496,  0.2838347 ,  0.18521205,
         0.05922702, -0.21033153, -0.20717393, -0.11639898,  0.037677  ,
        -0.35412446,  0.28322875, -0.22257015,  0.19381312,  0.13257173,
        -0.20125555,  0.3308853 , -0.16780548, -0.31859216,  0.27076027,
         0.31858438, -0.06772649,  0.24243534,  0.33016163, -0.29998642,
        -0.2597285 , -0.04579486],
       [ 0.14743018, -0.11593532, -0.16910997,  0.04817718, -0.08704076,
        -0.28638428, -0.0695084 , -0.20624651, -0.0902727 , -0.35968247,
        -0.13577892, -0.3435246 ,  0.07125922, -0.06206191,  0.28400767,
         0.37250602, -0.22116509,  0.05206121, -0.09041598, -0.23407578,
        -0.13930109,  0.2527983 ,  0.2586641 , -0.17652327, -0.08032648,
        -0.1917453 , -0.12685478,  0.06995464,  0.17967616, -0.13169824,
         0.31765214,  0.15295997],
       [-0.0138146 , -0.02914308, -0.28513512, -0.14760508,  0.25594202,
        -0.14606586, -0.0786619 ,  0.08240923, -0.19649631,  0.20795472,
        -0.02518724,  0.08480912,  0.14874646, -0.09301162, -0.31286716,
        -0.30428803, -0.28728503,  0.13118117,  0.12566496,  0.36451495,
        -0.06084813, -0.2617515 , -0.33376834,  0.10222124, -0.07494642,
         0.01660871, -0.07462687, -0.26671436,  0.05951952,  0.19430834,
         0.2599313 ,  0.18841784],
       [ 0.03526726,  0.0700261 , -0.14811565, -0.00161783,  0.24595922,
         0.22936851, -0.13471882,  0.30484498, -0.09300318,  0.1434963 ,
         0.34645033, -0.14074416, -0.13980703, -0.0728282 ,  0.05654244,
        -0.03714944, -0.02068539,  0.25234953, -0.12365915, -0.08969368,
         0.23459183, -0.3667168 , -0.10735053, -0.02297385,  0.14225936,
        -0.03950644,  0.00390262,  0.02620285, -0.23849073, -0.17826776,
        -0.00452239, -0.21916436],
       [ 0.08131967, -0.30762494,  0.13485682,  0.13356149,  0.34811306,
        -0.06675722, -0.26377225, -0.30782035, -0.10946912, -0.1001946 ,
        -0.00216706,  0.34092563,  0.20830967,  0.09819086,  0.10027841,
        -0.1561837 , -0.08846258,  0.00475864,  0.13237843, -0.07028957,
         0.0286342 ,  0.03975904,  0.329702  ,  0.2488215 , -0.1448916 ,
         0.04351248, -0.05134839,  0.11396249,  0.15381822,  0.22561055,
        -0.22243582, -0.2165636 ],
       [ 0.1493244 , -0.01058014,  0.09766275,  0.27777147, -0.22158019,
         0.18286034, -0.08336566,  0.13315946,  0.25735724,  0.30913785,
         0.24050163,  0.03009882, -0.21949081, -0.03347325, -0.23332547,
         0.09693161,  0.02377792, -0.02950464, -0.24924129, -0.27218467,
        -0.30237398, -0.04576043,  0.09367327, -0.00097414,  0.04334731,
         0.23533046,  0.10836121, -0.22805499, -0.35266235,  0.2814061 ,
        -0.26131305,  0.20472598],
       [-0.14326951,  0.21294083, -0.01614356, -0.08817112, -0.05104741,
        -0.23520012,  0.30071735,  0.32639796, -0.25743112,  0.1611526 ,
        -0.31708157,  0.32357627, -0.0143376 ,  0.05381984, -0.2437209 ,
        -0.03047014,  0.15428443, -0.31354713, -0.19772638, -0.03241993,
         0.0728471 , -0.20864433, -0.14207917, -0.02163322,  0.0778316 ,
        -0.3518254 ,  0.13569653,  0.31376123,  0.02812677, -0.29967225,
         0.06654844,  0.02718871],
       [ 0.16195253, -0.18050618,  0.23937896, -0.24488612, -0.3065511 ,
         0.00333037,  0.3497712 , -0.0959979 ,  0.2496489 , -0.06721076,
        -0.04689456,  0.08144559,  0.25725022,  0.1737523 ,  0.33267274,
        -0.18413769,  0.22699034,  0.34987855, -0.18642148, -0.19489624,
         0.0458399 ,  0.2265654 ,  0.04685582, -0.007166  , -0.05132204,
        -0.29220864, -0.05268902, -0.24997792, -0.00162561, -0.14898796,
         0.1487238 , -0.13080698]], dtype=float32)
In [106]:
print(model_mlp.layers[0].bias.numpy())

[-0.06321038 -0.09043414 -0.03237854 -0.04901438  0.03637632 -0.00020139
 -0.00765608 -0.04771591 -0.02193414 -0.05995064 -0.02930783  0.02134661
  0.08075356 -0.02265028 -0.02011513  0.01451884 -0.00299843 -0.01681603
  0.0088689   0.09928493 -0.01004401 -0.04191288  0.03498737  0.0121335
 -0.01918527 -0.00528463  0.00847592  0.02848048 -0.02629845  0.04105879
  0.07961563  0.02241718]
In [113]:
mse_test=model_mlp.evaluate(X_test,y_test)
print(mse_test)

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 372ms/step - loss: 0.0160 - mean_squared_error: 0.0160
[0.015960311517119408, 0.015960311517119408]
In [107]:
X_test.shape
prediction_test=model_mlp.predict( X_test, verbose=1)

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 156ms/step
In [108]:
y_test=y_test.reshape((y_test.shape[0], 1))
In [109]:
prediction_test.shape
Out[109]:
(12, 1)
In [110]:
y_test.shape
Out[110]:
(12, 1)
In [111]:
from sklearn.metrics import mean_squared_error

errors_squared = mean_squared_error(y_test, prediction_test, squared=True)
rmse = errors_squared**0.5
print("RECM:", rmse)

RECM: 0.12633412302818484
In [112]:
import matplotlib.pyplot as plt

plt.figure(figsize=(10, 6))  # Tamaño de la figura

# Gráfica de los datos reales y las predicciones
plt.plot(y_test, label='Respuesta Real', marker='o', color = 'blue')  # Datos reales
plt.plot(prediction_test, label='Predicción de la Respuesta', linestyle='-', marker='o', color = 'orange')  # Predicciones

# Leyenda y etiquetas
plt.legend(fontsize=12)  # Leyenda con tamaño de fuente
plt.ylabel('Valores de Y y $\hat{Y}$ en conjunto de prueba', fontsize=14)  # Etiqueta del eje y

# Añadir título y ajustes adicionales
plt.title('Comparación entre Respuesta Real y Predicción', fontsize=16)  # Título de la gráfica
plt.xlabel('Índice de Muestra', fontsize=12)  # Etiqueta del eje x
plt.grid(True, linestyle='--', alpha=0.7)  # Cuadrícula punteada

plt.show()  # Mostrar la gráfica
No description has been provided for this image

Podemos observar que aplicando la busqueda de hiperparametros para la Red Neuronal Multicapa no hay un cambio notable en las predicciones qye se hacen sobre el conjunto de prueba, el cambio en el MSE es muy pequeño, lo cual corrobora lo que vemos en las graficas.

REDES NEURONALES RECURRENTE RNN

In [25]:
Serie = STS
STS.info
Out[25]:
<bound method Series.info of Fecha
2012-01-01   -0.184492
2012-02-01    0.107021
2012-03-01    0.336302
2012-04-01   -0.440628
2012-05-01    0.352852
                ...   
2023-08-01    0.169021
2023-09-01    0.165515
2023-10-01   -0.340459
2023-11-01    0.233495
2023-12-01   -0.062102
Name: BoxCox, Length: 144, dtype: float64>
In [26]:
df = pd.DataFrame(STS.values, index=STS.index, columns=['VAFODO'])
df
Out[26]:
VAFODO
Fecha
2012-01-01 -0.184492
2012-02-01 0.107021
2012-03-01 0.336302
2012-04-01 -0.440628
2012-05-01 0.352852
... ...
2023-08-01 0.169021
2023-09-01 0.165515
2023-10-01 -0.340459
2023-11-01 0.233495
2023-12-01 -0.062102

144 rows × 1 columns

In [27]:
nof_missing_values = sum(np.isnan(df['VAFODO']))

print(nof_missing_values, 'observations are missing.')
print('This is {:.3f}% of the total.'.format(nof_missing_values*100/len(df)))

0 observations are missing.
This is 0.000% of the total.
In [28]:
use_features =['VAFODO']  # continuous input
target = ['VAFODO'] # continuous output
n_steps_ahead = 1 # forecasting horizon
In [29]:
adf, p, usedlag, nobs, cvs, aic = sm.tsa.stattools.adfuller(df['VAFODO'])
adf_results_string = 'ADF: {}\np-value: {},\nN: {}, \ncritical values: {}'
print(adf_results_string.format(adf, p, nobs, cvs))

ADF: -6.050994014083185
p-value: 1.2777236623438134e-07,
N: 129, 
critical values: {'1%': -3.482087964046026, '5%': -2.8842185101614626, '10%': -2.578864381347275}

CONJUNTO DE ENTRENAMIENTO Y DE PRUEBA

In [30]:
train_weight = 0.8
split = int(len(df) * train_weight)

df_train = df[use_features].iloc[:split]
df_test = df[use_features].iloc[split:]
In [32]:
## ESCALADO 
mu = float(df_train.mean())
sigma = float(df_train.std())

stdize_input = lambda x: (x - mu) / sigma

df_train = df_train.apply(stdize_input)
df_test = df_test.apply(stdize_input)
In [39]:
def get_lagged_features(df, n_steps_ahead):
    """
    df: pandas DataFrame of time series to be lagged
    n_steps_ahead: forecasting horizon
    """
    lag_list = []
    
    # Agregar los primeros 3 rezagos
    for lag in range(3, 0, -1):
        lag_list.append(df.shift(lag))
        
    # Agregar rezagos que son múltiplos de 3 hasta 24
    for lag in range(3, 25, 3):
        lag_list.append(df.shift(lag))
        
    # Crear el arreglo tridimensional
    lag_array = np.dstack([i[max(3, 24)+n_steps_ahead-1:] for i in lag_list])
    lag_array = np.swapaxes(lag_array, 1, -1)
    return lag_array
In [36]:
n_steps
Out[36]:
1
In [61]:
x_train = get_lagged_features(df_train, n_steps_ahead)
y_train = df_train.values[n_steps_ahead + max(3, 24) - 1:]  # Recortar y_train
y_train_timestamps = df_train.index[n_steps_ahead + max(3, 24) - 1:]

x_test = get_lagged_features(df_test, n_steps_ahead)
y_test = df_test.values[n_steps_ahead + max(3, 24) - 1:]  # Recortar y_test
y_test_timestamps = df_test.index[n_steps_ahead + max(3, 24) - 1:]
In [62]:
print([tensor.shape for tensor in (x_train, y_train, x_test, y_test)])

[(91, 11, 1), (91, 1), (5, 11, 1), (5, 1)]

ESPECIFICACIONES DEL MODELO

In [43]:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import statsmodels.api as sm
import tensorflow as tf
from datetime import timedelta

from sklearn.metrics import mean_squared_error
from sklearn.model_selection import KFold, TimeSeriesSplit, GridSearchCV

import tensorflow.keras as keras
In [54]:
def SimpleRNN_(n_units = 10, l1_reg=0.0, seed=0):
  model = keras.models.Sequential()
  model.add(keras.layers.SimpleRNN(n_units, activation='tanh', kernel_initializer=keras.initializers.glorot_uniform(seed), bias_initializer=keras.initializers.glorot_uniform(seed), recurrent_initializer=keras.initializers.orthogonal(seed), kernel_regularizer=keras.regularizers.l1(l1_reg), input_shape=(x_train.shape[1], x_train.shape[-1]), unroll=True, stateful=False))  
  model.add(keras.layers.Dense(1, kernel_initializer=keras.initializers.glorot_uniform(seed), bias_initializer=keras.initializers.glorot_uniform(seed), kernel_regularizer=keras.regularizers.l1(l1_reg)))
  model.compile(loss='mean_squared_error', optimizer='adam')
  return model


def GRU_(n_units = 10, l1_reg=0.0, seed=0):
  model = keras.models.Sequential()
  model.add(keras.layers.GRU(n_units, activation='tanh', kernel_initializer=keras.initializers.glorot_uniform(seed), bias_initializer=keras.initializers.glorot_uniform(seed), recurrent_initializer=keras.initializers.orthogonal(seed), kernel_regularizer=keras.regularizers.l1(l1_reg), input_shape=(x_train.shape[1], x_train.shape[-1]), unroll=True))  
  model.add(keras.layers.Dense(1, kernel_initializer=keras.initializers.glorot_uniform(seed), bias_initializer=keras.initializers.glorot_uniform(seed), kernel_regularizer=keras.regularizers.l1(l1_reg)))
  model.compile(loss='mean_squared_error', optimizer='adam')
  return model


def LSTM_(n_units = 10, l1_reg=0.0, seed=0):
  model = keras.models.Sequential()
  model.add(keras.layers.LSTM(n_units, activation='tanh', kernel_initializer=keras.initializers.glorot_uniform(seed), bias_initializer=keras.initializers.glorot_uniform(seed), recurrent_initializer=keras.initializers.orthogonal(seed), kernel_regularizer=keras.regularizers.l1(l1_reg), input_shape=(x_train.shape[1], x_train.shape[-1]), unroll=True)) 
  model.add(keras.layers.Dense(1, kernel_initializer=keras.initializers.glorot_uniform(seed), bias_initializer=keras.initializers.glorot_uniform(seed), kernel_regularizer=keras.regularizers.l1(l1_reg)))
  model.compile(loss='mean_squared_error', optimizer='adam')
  return model
In [53]:
max_epochs = 20
batch_size = 10
In [55]:
es = keras.callbacks.EarlyStopping(monitor='loss', mode='min', verbose=1, patience=10, min_delta=1e-7, restore_best_weights=True)
In [56]:
params = {
    'rnn': {
        'model': None, 'function': SimpleRNN_, 'l1_reg': 0.0, 'H': 20, 
        'color': 'blue', 'label':'RNN'}, 
    'gru': {
        'model': None, 'function':GRU_,'l1_reg': 0.0, 'H': 10, 
        'color': 'orange', 'label': 'GRU'},
    'lstm': {
        'model': None, 'function': LSTM_,'l1_reg': 0.0, 'H': 10, 
        'color':'red', 'label': 'LSTM'}
}
In [57]:
do_training = True
In [58]:
if do_training is False:
     for key in params.keys():
        params[key]['model']=keras.models.load_model('RNNs-IMPORTACIONES-VAFODO-SAVED-' + key + '.hdf5')

VALIDACION CRUZADA¶

In [59]:
from scikeras.wrappers import KerasRegressor
from sklearn.model_selection import GridSearchCV, TimeSeriesSplit
In [63]:
cross_val = True # WARNING: Changing this to True will take many hours to run

if do_training and cross_val:
    n_units = [5, 10, 20]
    l1_reg = [0, 0.001, 0.01, 0.1]
    
    # A dictionary containing a list of values to be iterated through
    # for each parameter of the model included in the search
    param_grid = {'n_units': n_units, 'l1_reg': l1_reg}
    
    # In the kth split, TimeSeriesSplit returns first k folds 
    # as training set and the (k+1)th fold as test set.
    tscv = TimeSeriesSplit(n_splits = 5)
    
    # A grid search is performed for each of the models, and the parameter set which
    # performs best over all the cross-validation splits is saved in the `params` dictionary
    for key in params.keys():
        print('Performing cross-validation. Model:', key)
        model = KerasRegressor(model=params[key]['function'], epochs=max_epochs, 
                               batch_size=batch_size, verbose=2, l1_reg=0, n_units=10)
        #from scikeras.wrappers import KerasClassifier, KerasRegressor
        grid = GridSearchCV(estimator=model, param_grid=param_grid, 
                            cv=tscv, n_jobs=1, verbose=2)
        grid_result = grid.fit(x_train, y_train, callbacks=[es])
        print("Best: %f using %s" % (grid_result.best_score_, grid_result.best_params_))
        
        means = grid_result.cv_results_['mean_test_score']
        stds = grid_result.cv_results_['std_test_score']
        params_ = grid_result.cv_results_['params']
        for mean, stdev, param_ in zip(means, stds, params_):
            print("%f (%f) with %r" % (mean, stdev, param_))
            
        params[key]['H'] = grid_result.best_params_['n_units']
        params[key]['l1_reg']= grid_result.best_params_['l1_reg']

Performing cross-validation. Model: rnn
Fitting 5 folds for each of 12 candidates, totalling 60 fits
Epoch 1/20
[CV] END ................................l1_reg=0, n_units=5; total time=   0.5s
Epoch 1/20
[CV] END ................................l1_reg=0, n_units=5; total time=   0.1s
Epoch 1/20
[CV] END ................................l1_reg=0, n_units=5; total time=   0.0s
Epoch 1/20
[CV] END ................................l1_reg=0, n_units=5; total time=   0.0s
Epoch 1/20
[CV] END ................................l1_reg=0, n_units=5; total time=   0.0s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=10; total time=   0.1s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=10; total time=   0.1s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=10; total time=   0.1s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=10; total time=   0.1s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=10; total time=   0.1s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=20; total time=   0.1s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=20; total time=   0.1s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=20; total time=   0.1s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=20; total time=   0.1s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=20; total time=   0.1s
Epoch 1/20
2/2 - 1s - 554ms/step - loss: 3.1290
Epoch 2/20
2/2 - 0s - 15ms/step - loss: 3.1072
Epoch 3/20
2/2 - 0s - 17ms/step - loss: 3.0844
Epoch 4/20
2/2 - 0s - 10ms/step - loss: 3.0661
Epoch 5/20
2/2 - 0s - 13ms/step - loss: 3.0392
Epoch 6/20
2/2 - 0s - 13ms/step - loss: 3.0187
Epoch 7/20
2/2 - 0s - 15ms/step - loss: 2.9936
Epoch 8/20
2/2 - 0s - 14ms/step - loss: 2.9757
Epoch 9/20
2/2 - 0s - 17ms/step - loss: 2.9545
Epoch 10/20
2/2 - 0s - 12ms/step - loss: 2.9291
Epoch 11/20
2/2 - 0s - 18ms/step - loss: 2.9128
Epoch 12/20
2/2 - 0s - 13ms/step - loss: 2.8925
Epoch 13/20
2/2 - 0s - 18ms/step - loss: 2.8746
Epoch 14/20
2/2 - 0s - 16ms/step - loss: 2.8554
Epoch 15/20
2/2 - 0s - 16ms/step - loss: 2.8360
Epoch 16/20
2/2 - 0s - 18ms/step - loss: 2.8180
Epoch 17/20
2/2 - 0s - 17ms/step - loss: 2.8004
Epoch 18/20
2/2 - 0s - 15ms/step - loss: 2.7841
Epoch 19/20
2/2 - 0s - 14ms/step - loss: 2.7690
Epoch 20/20
2/2 - 0s - 15ms/step - loss: 2.7522
2/2 - 0s - 123ms/step
[CV] END ............................l1_reg=0.001, n_units=5; total time=   2.0s
Epoch 1/20
4/4 - 1s - 247ms/step - loss: 2.8537
Epoch 2/20
4/4 - 0s - 7ms/step - loss: 2.8253
Epoch 3/20
4/4 - 0s - 7ms/step - loss: 2.7952
Epoch 4/20
4/4 - 0s - 8ms/step - loss: 2.7683
Epoch 5/20
4/4 - 0s - 6ms/step - loss: 2.7443
Epoch 6/20
4/4 - 0s - 9ms/step - loss: 2.7228
Epoch 7/20
4/4 - 0s - 10ms/step - loss: 2.6989
Epoch 8/20
4/4 - 0s - 7ms/step - loss: 2.6814
Epoch 9/20
4/4 - 0s - 8ms/step - loss: 2.6573
Epoch 10/20
4/4 - 0s - 24ms/step - loss: 2.6371
Epoch 11/20
4/4 - 0s - 7ms/step - loss: 2.6166
Epoch 12/20
4/4 - 0s - 8ms/step - loss: 2.5959
Epoch 13/20
4/4 - 0s - 10ms/step - loss: 2.5781
Epoch 14/20
4/4 - 0s - 10ms/step - loss: 2.5618
Epoch 15/20
4/4 - 0s - 9ms/step - loss: 2.5412
Epoch 16/20
4/4 - 0s - 10ms/step - loss: 2.5226
Epoch 17/20
4/4 - 0s - 8ms/step - loss: 2.5061
Epoch 18/20
4/4 - 0s - 7ms/step - loss: 2.4850
Epoch 19/20
4/4 - 0s - 7ms/step - loss: 2.4654
Epoch 20/20
4/4 - 0s - 9ms/step - loss: 2.4476
2/2 - 0s - 126ms/step
[CV] END ............................l1_reg=0.001, n_units=5; total time=   1.9s
Epoch 1/20
5/5 - 1s - 203ms/step - loss: 2.4348
Epoch 2/20
5/5 - 0s - 6ms/step - loss: 2.3946
Epoch 3/20
5/5 - 0s - 5ms/step - loss: 2.3579
Epoch 4/20
5/5 - 0s - 5ms/step - loss: 2.3183
Epoch 5/20
5/5 - 0s - 6ms/step - loss: 2.2830
Epoch 6/20
5/5 - 0s - 6ms/step - loss: 2.2484
Epoch 7/20
5/5 - 0s - 5ms/step - loss: 2.2158
Epoch 8/20
5/5 - 0s - 5ms/step - loss: 2.1860
Epoch 9/20
5/5 - 0s - 6ms/step - loss: 2.1504
Epoch 10/20
5/5 - 0s - 6ms/step - loss: 2.1209
Epoch 11/20
5/5 - 0s - 6ms/step - loss: 2.0890
Epoch 12/20
5/5 - 0s - 7ms/step - loss: 2.0599
Epoch 13/20
5/5 - 0s - 7ms/step - loss: 2.0335
Epoch 14/20
5/5 - 0s - 5ms/step - loss: 2.0087
Epoch 15/20
5/5 - 0s - 6ms/step - loss: 1.9851
Epoch 16/20
5/5 - 0s - 8ms/step - loss: 1.9578
Epoch 17/20
5/5 - 0s - 6ms/step - loss: 1.9362
Epoch 18/20
5/5 - 0s - 6ms/step - loss: 1.9124
Epoch 19/20
5/5 - 0s - 7ms/step - loss: 1.8909
Epoch 20/20
5/5 - 0s - 5ms/step - loss: 1.8693
WARNING:tensorflow:5 out of the last 5 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x0000016D1F3D8CC0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING:tensorflow:6 out of the last 6 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x0000016D1F3D8CC0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
2/2 - 0s - 118ms/step
[CV] END ............................l1_reg=0.001, n_units=5; total time=   1.9s
Epoch 1/20
7/7 - 1s - 142ms/step - loss: 2.4766
Epoch 2/20
7/7 - 0s - 5ms/step - loss: 2.4180
Epoch 3/20
7/7 - 0s - 5ms/step - loss: 2.3658
Epoch 4/20
7/7 - 0s - 4ms/step - loss: 2.3142
Epoch 5/20
7/7 - 0s - 4ms/step - loss: 2.2681
Epoch 6/20
7/7 - 0s - 5ms/step - loss: 2.2170
Epoch 7/20
7/7 - 0s - 4ms/step - loss: 2.1727
Epoch 8/20
7/7 - 0s - 5ms/step - loss: 2.1326
Epoch 9/20
7/7 - 0s - 4ms/step - loss: 2.0973
Epoch 10/20
7/7 - 0s - 4ms/step - loss: 2.0617
Epoch 11/20
7/7 - 0s - 4ms/step - loss: 2.0300
Epoch 12/20
7/7 - 0s - 5ms/step - loss: 1.9953
Epoch 13/20
7/7 - 0s - 4ms/step - loss: 1.9632
Epoch 14/20
7/7 - 0s - 5ms/step - loss: 1.9336
Epoch 15/20
7/7 - 0s - 14ms/step - loss: 1.8956
Epoch 16/20
7/7 - 0s - 5ms/step - loss: 1.8641
Epoch 17/20
7/7 - 0s - 4ms/step - loss: 1.8396
Epoch 18/20
7/7 - 0s - 5ms/step - loss: 1.8044
Epoch 19/20
7/7 - 0s - 5ms/step - loss: 1.7808
Epoch 20/20
7/7 - 0s - 6ms/step - loss: 1.7541
2/2 - 0s - 113ms/step
[CV] END ............................l1_reg=0.001, n_units=5; total time=   1.9s
Epoch 1/20
8/8 - 1s - 132ms/step - loss: 2.1565
Epoch 2/20
8/8 - 0s - 4ms/step - loss: 2.0988
Epoch 3/20
8/8 - 0s - 4ms/step - loss: 2.0419
Epoch 4/20
8/8 - 0s - 4ms/step - loss: 1.9865
Epoch 5/20
8/8 - 0s - 3ms/step - loss: 1.9360
Epoch 6/20
8/8 - 0s - 4ms/step - loss: 1.8927
Epoch 7/20
8/8 - 0s - 4ms/step - loss: 1.8484
Epoch 8/20
8/8 - 0s - 4ms/step - loss: 1.8041
Epoch 9/20
8/8 - 0s - 4ms/step - loss: 1.7655
Epoch 10/20
8/8 - 0s - 4ms/step - loss: 1.7266
Epoch 11/20
8/8 - 0s - 4ms/step - loss: 1.6920
Epoch 12/20
8/8 - 0s - 4ms/step - loss: 1.6547
Epoch 13/20
8/8 - 0s - 5ms/step - loss: 1.6208
Epoch 14/20
8/8 - 0s - 5ms/step - loss: 1.5888
Epoch 15/20
8/8 - 0s - 4ms/step - loss: 1.5602
Epoch 16/20
8/8 - 0s - 4ms/step - loss: 1.5293
Epoch 17/20
8/8 - 0s - 4ms/step - loss: 1.5001
Epoch 18/20
8/8 - 0s - 4ms/step - loss: 1.4772
Epoch 19/20
8/8 - 0s - 4ms/step - loss: 1.4497
Epoch 20/20
8/8 - 0s - 4ms/step - loss: 1.4246
2/2 - 0s - 123ms/step
[CV] END ............................l1_reg=0.001, n_units=5; total time=   1.9s
Epoch 1/20
2/2 - 1s - 625ms/step - loss: 4.5202
Epoch 2/20
2/2 - 0s - 14ms/step - loss: 4.4502
Epoch 3/20
2/2 - 0s - 10ms/step - loss: 4.4020
Epoch 4/20
2/2 - 0s - 13ms/step - loss: 4.3408
Epoch 5/20
2/2 - 0s - 10ms/step - loss: 4.2907
Epoch 6/20
2/2 - 0s - 13ms/step - loss: 4.2302
Epoch 7/20
2/2 - 0s - 11ms/step - loss: 4.1784
Epoch 8/20
2/2 - 0s - 15ms/step - loss: 4.1277
Epoch 9/20
2/2 - 0s - 12ms/step - loss: 4.0790
Epoch 10/20
2/2 - 0s - 13ms/step - loss: 4.0273
Epoch 11/20
2/2 - 0s - 14ms/step - loss: 3.9790
Epoch 12/20
2/2 - 0s - 14ms/step - loss: 3.9343
Epoch 13/20
2/2 - 0s - 12ms/step - loss: 3.8902
Epoch 14/20
2/2 - 0s - 14ms/step - loss: 3.8429
Epoch 15/20
2/2 - 0s - 14ms/step - loss: 3.8035
Epoch 16/20
2/2 - 0s - 14ms/step - loss: 3.7634
Epoch 17/20
2/2 - 0s - 15ms/step - loss: 3.7220
Epoch 18/20
2/2 - 0s - 12ms/step - loss: 3.6853
Epoch 19/20
2/2 - 0s - 16ms/step - loss: 3.6438
Epoch 20/20
2/2 - 0s - 13ms/step - loss: 3.6128
2/2 - 0s - 121ms/step
[CV] END ...........................l1_reg=0.001, n_units=10; total time=   2.0s
Epoch 1/20
4/4 - 1s - 259ms/step - loss: 4.0042
Epoch 2/20
4/4 - 0s - 7ms/step - loss: 3.9020
Epoch 3/20
4/4 - 0s - 5ms/step - loss: 3.8194
Epoch 4/20
4/4 - 0s - 6ms/step - loss: 3.7319
Epoch 5/20
4/4 - 0s - 6ms/step - loss: 3.6444
Epoch 6/20
4/4 - 0s - 7ms/step - loss: 3.5756
Epoch 7/20
4/4 - 0s - 6ms/step - loss: 3.5036
Epoch 8/20
4/4 - 0s - 6ms/step - loss: 3.4450
Epoch 9/20
4/4 - 0s - 7ms/step - loss: 3.3825
Epoch 10/20
4/4 - 0s - 6ms/step - loss: 3.3267
Epoch 11/20
4/4 - 0s - 6ms/step - loss: 3.2771
Epoch 12/20
4/4 - 0s - 7ms/step - loss: 3.2302
Epoch 13/20
4/4 - 0s - 6ms/step - loss: 3.1806
Epoch 14/20
4/4 - 0s - 7ms/step - loss: 3.1433
Epoch 15/20
4/4 - 0s - 6ms/step - loss: 3.0937
Epoch 16/20
4/4 - 0s - 7ms/step - loss: 3.0542
Epoch 17/20
4/4 - 0s - 8ms/step - loss: 3.0159
Epoch 18/20
4/4 - 0s - 8ms/step - loss: 2.9804
Epoch 19/20
4/4 - 0s - 6ms/step - loss: 2.9433
Epoch 20/20
4/4 - 0s - 7ms/step - loss: 2.9133
2/2 - 0s - 123ms/step
[CV] END ...........................l1_reg=0.001, n_units=10; total time=   1.8s
Epoch 1/20
5/5 - 1s - 210ms/step - loss: 3.4825
Epoch 2/20
5/5 - 0s - 5ms/step - loss: 3.3740
Epoch 3/20
5/5 - 0s - 6ms/step - loss: 3.2816
Epoch 4/20
5/5 - 0s - 6ms/step - loss: 3.1890
Epoch 5/20
5/5 - 0s - 5ms/step - loss: 3.0983
Epoch 6/20
5/5 - 0s - 6ms/step - loss: 3.0299
Epoch 7/20
5/5 - 0s - 6ms/step - loss: 2.9565
Epoch 8/20
5/5 - 0s - 5ms/step - loss: 2.8906
Epoch 9/20
5/5 - 0s - 6ms/step - loss: 2.8299
Epoch 10/20
5/5 - 0s - 5ms/step - loss: 2.7620
Epoch 11/20
5/5 - 0s - 6ms/step - loss: 2.7149
Epoch 12/20
5/5 - 0s - 5ms/step - loss: 2.6527
Epoch 13/20
5/5 - 0s - 6ms/step - loss: 2.5954
Epoch 14/20
5/5 - 0s - 8ms/step - loss: 2.5486
Epoch 15/20
5/5 - 0s - 6ms/step - loss: 2.5037
Epoch 16/20
5/5 - 0s - 5ms/step - loss: 2.4531
Epoch 17/20
5/5 - 0s - 5ms/step - loss: 2.4128
Epoch 18/20
5/5 - 0s - 6ms/step - loss: 2.3661
Epoch 19/20
5/5 - 0s - 5ms/step - loss: 2.3265
Epoch 20/20
5/5 - 0s - 6ms/step - loss: 2.2842
2/2 - 0s - 122ms/step
[CV] END ...........................l1_reg=0.001, n_units=10; total time=   1.9s
Epoch 1/20
7/7 - 1s - 146ms/step - loss: 3.5490
Epoch 2/20
7/7 - 0s - 5ms/step - loss: 3.4225
Epoch 3/20
7/7 - 0s - 5ms/step - loss: 3.3143
Epoch 4/20
7/7 - 0s - 4ms/step - loss: 3.2017
Epoch 5/20
7/7 - 0s - 4ms/step - loss: 3.0960
Epoch 6/20
7/7 - 0s - 5ms/step - loss: 3.0110
Epoch 7/20
7/7 - 0s - 4ms/step - loss: 2.9256
Epoch 8/20
7/7 - 0s - 4ms/step - loss: 2.8491
Epoch 9/20
7/7 - 0s - 5ms/step - loss: 2.7724
Epoch 10/20
7/7 - 0s - 5ms/step - loss: 2.6993
Epoch 11/20
7/7 - 0s - 4ms/step - loss: 2.6347
Epoch 12/20
7/7 - 0s - 5ms/step - loss: 2.5668
Epoch 13/20
7/7 - 0s - 5ms/step - loss: 2.5122
Epoch 14/20
7/7 - 0s - 4ms/step - loss: 2.4576
Epoch 15/20
7/7 - 0s - 5ms/step - loss: 2.3992
Epoch 16/20
7/7 - 0s - 6ms/step - loss: 2.3467
Epoch 17/20
7/7 - 0s - 6ms/step - loss: 2.2976
Epoch 18/20
7/7 - 0s - 5ms/step - loss: 2.2507
Epoch 19/20
7/7 - 0s - 5ms/step - loss: 2.2067
Epoch 20/20
7/7 - 0s - 5ms/step - loss: 2.1628
2/2 - 0s - 117ms/step
[CV] END ...........................l1_reg=0.001, n_units=10; total time=   1.9s
Epoch 1/20
8/8 - 1s - 129ms/step - loss: 3.0933
Epoch 2/20
8/8 - 0s - 5ms/step - loss: 2.9650
Epoch 3/20
8/8 - 0s - 4ms/step - loss: 2.8379
Epoch 4/20
8/8 - 0s - 4ms/step - loss: 2.7330
Epoch 5/20
8/8 - 0s - 4ms/step - loss: 2.6288
Epoch 6/20
8/8 - 0s - 4ms/step - loss: 2.5310
Epoch 7/20
8/8 - 0s - 5ms/step - loss: 2.4433
Epoch 8/20
8/8 - 0s - 4ms/step - loss: 2.3570
Epoch 9/20
8/8 - 0s - 4ms/step - loss: 2.2715
Epoch 10/20
8/8 - 0s - 5ms/step - loss: 2.1994
Epoch 11/20
8/8 - 0s - 5ms/step - loss: 2.1278
Epoch 12/20
8/8 - 0s - 7ms/step - loss: 2.0579
Epoch 13/20
8/8 - 0s - 5ms/step - loss: 1.9932
Epoch 14/20
8/8 - 0s - 4ms/step - loss: 1.9339
Epoch 15/20
8/8 - 0s - 6ms/step - loss: 1.8688
Epoch 16/20
8/8 - 0s - 6ms/step - loss: 1.8167
Epoch 17/20
8/8 - 0s - 5ms/step - loss: 1.7661
Epoch 18/20
8/8 - 0s - 5ms/step - loss: 1.7181
Epoch 19/20
8/8 - 0s - 5ms/step - loss: 1.6658
Epoch 20/20
8/8 - 0s - 5ms/step - loss: 1.6237
2/2 - 0s - 133ms/step
[CV] END ...........................l1_reg=0.001, n_units=10; total time=   2.0s
Epoch 1/20
2/2 - 1s - 496ms/step - loss: 5.3865
Epoch 2/20
2/2 - 0s - 13ms/step - loss: 5.2374
Epoch 3/20
2/2 - 0s - 11ms/step - loss: 5.1088
Epoch 4/20
2/2 - 0s - 13ms/step - loss: 4.9742
Epoch 5/20
2/2 - 0s - 11ms/step - loss: 4.8644
Epoch 6/20
2/2 - 0s - 12ms/step - loss: 4.7685
Epoch 7/20
2/2 - 0s - 14ms/step - loss: 4.6690
Epoch 8/20
2/2 - 0s - 11ms/step - loss: 4.5760
Epoch 9/20
2/2 - 0s - 13ms/step - loss: 4.4767
Epoch 10/20
2/2 - 0s - 16ms/step - loss: 4.3999
Epoch 11/20
2/2 - 0s - 17ms/step - loss: 4.3305
Epoch 12/20
2/2 - 0s - 17ms/step - loss: 4.2543
Epoch 13/20
2/2 - 0s - 15ms/step - loss: 4.1774
Epoch 14/20
2/2 - 0s - 13ms/step - loss: 4.1312
Epoch 15/20
2/2 - 0s - 12ms/step - loss: 4.0554
Epoch 16/20
2/2 - 0s - 12ms/step - loss: 4.0078
Epoch 17/20
2/2 - 0s - 14ms/step - loss: 3.9662
Epoch 18/20
2/2 - 0s - 17ms/step - loss: 3.9011
Epoch 19/20
2/2 - 0s - 17ms/step - loss: 3.8574
Epoch 20/20
2/2 - 0s - 15ms/step - loss: 3.8095
2/2 - 0s - 120ms/step
[CV] END ...........................l1_reg=0.001, n_units=20; total time=   1.8s
Epoch 1/20
4/4 - 1s - 297ms/step - loss: 4.6801
Epoch 2/20
4/4 - 0s - 7ms/step - loss: 4.4881
Epoch 3/20
4/4 - 0s - 7ms/step - loss: 4.3124
Epoch 4/20
4/4 - 0s - 6ms/step - loss: 4.1678
Epoch 5/20
4/4 - 0s - 6ms/step - loss: 4.0371
Epoch 6/20
4/4 - 0s - 7ms/step - loss: 3.9204
Epoch 7/20
4/4 - 0s - 7ms/step - loss: 3.8159
Epoch 8/20
4/4 - 0s - 9ms/step - loss: 3.7167
Epoch 9/20
4/4 - 0s - 7ms/step - loss: 3.6263
Epoch 10/20
4/4 - 0s - 8ms/step - loss: 3.5378
Epoch 11/20
4/4 - 0s - 8ms/step - loss: 3.4707
Epoch 12/20
4/4 - 0s - 7ms/step - loss: 3.3938
Epoch 13/20
4/4 - 0s - 10ms/step - loss: 3.3346
Epoch 14/20
4/4 - 0s - 8ms/step - loss: 3.2624
Epoch 15/20
4/4 - 0s - 7ms/step - loss: 3.1841
Epoch 16/20
4/4 - 0s - 8ms/step - loss: 3.1220
Epoch 17/20
4/4 - 0s - 7ms/step - loss: 3.0658
Epoch 18/20
4/4 - 0s - 7ms/step - loss: 3.0131
Epoch 19/20
4/4 - 0s - 9ms/step - loss: 2.9605
Epoch 20/20
4/4 - 0s - 9ms/step - loss: 2.9234
2/2 - 0s - 128ms/step
[CV] END ...........................l1_reg=0.001, n_units=20; total time=   2.1s
Epoch 1/20
5/5 - 1s - 296ms/step - loss: 4.1004
Epoch 2/20
5/5 - 0s - 8ms/step - loss: 3.9068
Epoch 3/20
5/5 - 0s - 7ms/step - loss: 3.7246
Epoch 4/20
5/5 - 0s - 6ms/step - loss: 3.6000
Epoch 5/20
5/5 - 0s - 7ms/step - loss: 3.4679
Epoch 6/20
5/5 - 0s - 6ms/step - loss: 3.3505
Epoch 7/20
5/5 - 0s - 7ms/step - loss: 3.2534
Epoch 8/20
5/5 - 0s - 6ms/step - loss: 3.1449
Epoch 9/20
5/5 - 0s - 8ms/step - loss: 3.0573
Epoch 10/20
5/5 - 0s - 7ms/step - loss: 2.9563
Epoch 11/20
5/5 - 0s - 6ms/step - loss: 2.8792
Epoch 12/20
5/5 - 0s - 7ms/step - loss: 2.8009
Epoch 13/20
5/5 - 0s - 6ms/step - loss: 2.7224
Epoch 14/20
5/5 - 0s - 5ms/step - loss: 2.6518
Epoch 15/20
5/5 - 0s - 6ms/step - loss: 2.5805
Epoch 16/20
5/5 - 0s - 7ms/step - loss: 2.5188
Epoch 17/20
5/5 - 0s - 7ms/step - loss: 2.4588
Epoch 18/20
5/5 - 0s - 7ms/step - loss: 2.3944
Epoch 19/20
5/5 - 0s - 7ms/step - loss: 2.3469
Epoch 20/20
5/5 - 0s - 9ms/step - loss: 2.2901
2/2 - 0s - 167ms/step
[CV] END ...........................l1_reg=0.001, n_units=20; total time=   2.5s
Epoch 1/20
7/7 - 1s - 192ms/step - loss: 4.2023
Epoch 2/20
7/7 - 0s - 5ms/step - loss: 3.9330
Epoch 3/20
7/7 - 0s - 5ms/step - loss: 3.7021
Epoch 4/20
7/7 - 0s - 5ms/step - loss: 3.5275
Epoch 5/20
7/7 - 0s - 5ms/step - loss: 3.3564
Epoch 6/20
7/7 - 0s - 6ms/step - loss: 3.2093
Epoch 7/20
7/7 - 0s - 6ms/step - loss: 3.0775
Epoch 8/20
7/7 - 0s - 6ms/step - loss: 2.9672
Epoch 9/20
7/7 - 0s - 5ms/step - loss: 2.8639
Epoch 10/20
7/7 - 0s - 6ms/step - loss: 2.7808
Epoch 11/20
7/7 - 0s - 5ms/step - loss: 2.6928
Epoch 12/20
7/7 - 0s - 6ms/step - loss: 2.6118
Epoch 13/20
7/7 - 0s - 7ms/step - loss: 2.5267
Epoch 14/20
7/7 - 0s - 5ms/step - loss: 2.4402
Epoch 15/20
7/7 - 0s - 5ms/step - loss: 2.3602
Epoch 16/20
7/7 - 0s - 4ms/step - loss: 2.2857
Epoch 17/20
7/7 - 0s - 4ms/step - loss: 2.2222
Epoch 18/20
7/7 - 0s - 6ms/step - loss: 2.1494
Epoch 19/20
7/7 - 0s - 8ms/step - loss: 2.0875
Epoch 20/20
7/7 - 0s - 4ms/step - loss: 2.0242
2/2 - 0s - 166ms/step
[CV] END ...........................l1_reg=0.001, n_units=20; total time=   2.4s
Epoch 1/20
8/8 - 1s - 138ms/step - loss: 3.6701
Epoch 2/20
8/8 - 0s - 4ms/step - loss: 3.3941
Epoch 3/20
8/8 - 0s - 4ms/step - loss: 3.1862
Epoch 4/20
8/8 - 0s - 4ms/step - loss: 3.0158
Epoch 5/20
8/8 - 0s - 3ms/step - loss: 2.8365
Epoch 6/20
8/8 - 0s - 4ms/step - loss: 2.7007
Epoch 7/20
8/8 - 0s - 4ms/step - loss: 2.5504
Epoch 8/20
8/8 - 0s - 4ms/step - loss: 2.4346
Epoch 9/20
8/8 - 0s - 4ms/step - loss: 2.3116
Epoch 10/20
8/8 - 0s - 4ms/step - loss: 2.2139
Epoch 11/20
8/8 - 0s - 4ms/step - loss: 2.1097
Epoch 12/20
8/8 - 0s - 6ms/step - loss: 2.0294
Epoch 13/20
8/8 - 0s - 4ms/step - loss: 1.9387
Epoch 14/20
8/8 - 0s - 4ms/step - loss: 1.8675
Epoch 15/20
8/8 - 0s - 5ms/step - loss: 1.8067
Epoch 16/20
8/8 - 0s - 4ms/step - loss: 1.7388
Epoch 17/20
8/8 - 0s - 5ms/step - loss: 1.6821
Epoch 18/20
8/8 - 0s - 4ms/step - loss: 1.6271
Epoch 19/20
8/8 - 0s - 6ms/step - loss: 1.5791
Epoch 20/20
8/8 - 0s - 4ms/step - loss: 1.5351
2/2 - 1s - 658ms/step
[CV] END ...........................l1_reg=0.001, n_units=20; total time=   3.1s
Epoch 1/20
2/2 - 1s - 522ms/step - loss: 3.1669
Epoch 2/20
2/2 - 0s - 11ms/step - loss: 3.1425
Epoch 3/20
2/2 - 0s - 14ms/step - loss: 3.1179
Epoch 4/20
2/2 - 0s - 13ms/step - loss: 3.0978
Epoch 5/20
2/2 - 0s - 13ms/step - loss: 3.0746
Epoch 6/20
2/2 - 0s - 12ms/step - loss: 3.0534
Epoch 7/20
2/2 - 0s - 12ms/step - loss: 3.0309
Epoch 8/20
2/2 - 0s - 15ms/step - loss: 3.0092
Epoch 9/20
2/2 - 0s - 14ms/step - loss: 2.9894
Epoch 10/20
2/2 - 0s - 15ms/step - loss: 2.9671
Epoch 11/20
2/2 - 0s - 21ms/step - loss: 2.9479
Epoch 12/20
2/2 - 0s - 17ms/step - loss: 2.9281
Epoch 13/20
2/2 - 0s - 21ms/step - loss: 2.9067
Epoch 14/20
2/2 - 0s - 15ms/step - loss: 2.8893
Epoch 15/20
2/2 - 0s - 19ms/step - loss: 2.8701
Epoch 16/20
2/2 - 0s - 18ms/step - loss: 2.8521
Epoch 17/20
2/2 - 0s - 15ms/step - loss: 2.8337
Epoch 18/20
2/2 - 0s - 14ms/step - loss: 2.8162
Epoch 19/20
2/2 - 0s - 15ms/step - loss: 2.8007
Epoch 20/20
2/2 - 0s - 22ms/step - loss: 2.7797
2/2 - 0s - 130ms/step
[CV] END .............................l1_reg=0.01, n_units=5; total time=   1.9s
Epoch 1/20
4/4 - 1s - 266ms/step - loss: 2.8869
Epoch 2/20
4/4 - 0s - 6ms/step - loss: 2.8466
Epoch 3/20
4/4 - 0s - 6ms/step - loss: 2.8186
Epoch 4/20
4/4 - 0s - 7ms/step - loss: 2.7874
Epoch 5/20
4/4 - 0s - 9ms/step - loss: 2.7541
Epoch 6/20
4/4 - 0s - 7ms/step - loss: 2.7241
Epoch 7/20
4/4 - 0s - 7ms/step - loss: 2.6988
Epoch 8/20
4/4 - 0s - 8ms/step - loss: 2.6708
Epoch 9/20
4/4 - 0s - 7ms/step - loss: 2.6450
Epoch 10/20
4/4 - 0s - 7ms/step - loss: 2.6197
Epoch 11/20
4/4 - 0s - 9ms/step - loss: 2.5949
Epoch 12/20
4/4 - 0s - 8ms/step - loss: 2.5743
Epoch 13/20
4/4 - 0s - 7ms/step - loss: 2.5556
Epoch 14/20
4/4 - 0s - 6ms/step - loss: 2.5378
Epoch 15/20
4/4 - 0s - 6ms/step - loss: 2.5231
Epoch 16/20
4/4 - 0s - 8ms/step - loss: 2.5021
Epoch 17/20
4/4 - 0s - 8ms/step - loss: 2.4845
Epoch 18/20
4/4 - 0s - 10ms/step - loss: 2.4672
Epoch 19/20
4/4 - 0s - 7ms/step - loss: 2.4521
Epoch 20/20
4/4 - 0s - 7ms/step - loss: 2.4373
2/2 - 0s - 116ms/step
[CV] END .............................l1_reg=0.01, n_units=5; total time=   1.9s
Epoch 1/20
5/5 - 1s - 213ms/step - loss: 2.4775
Epoch 2/20
5/5 - 0s - 6ms/step - loss: 2.4421
Epoch 3/20
5/5 - 0s - 5ms/step - loss: 2.4029
Epoch 4/20
5/5 - 0s - 6ms/step - loss: 2.3711
Epoch 5/20
5/5 - 0s - 5ms/step - loss: 2.3329
Epoch 6/20
5/5 - 0s - 6ms/step - loss: 2.2971
Epoch 7/20
5/5 - 0s - 6ms/step - loss: 2.2599
Epoch 8/20
5/5 - 0s - 6ms/step - loss: 2.2333
Epoch 9/20
5/5 - 0s - 6ms/step - loss: 2.1990
Epoch 10/20
5/5 - 0s - 5ms/step - loss: 2.1655
Epoch 11/20
5/5 - 0s - 7ms/step - loss: 2.1438
Epoch 12/20
5/5 - 0s - 6ms/step - loss: 2.1146
Epoch 13/20
5/5 - 0s - 6ms/step - loss: 2.0837
Epoch 14/20
5/5 - 0s - 6ms/step - loss: 2.0620
Epoch 15/20
5/5 - 0s - 6ms/step - loss: 2.0353
Epoch 16/20
5/5 - 0s - 8ms/step - loss: 2.0116
Epoch 17/20
5/5 - 0s - 8ms/step - loss: 1.9868
Epoch 18/20
5/5 - 0s - 6ms/step - loss: 1.9654
Epoch 19/20
5/5 - 0s - 6ms/step - loss: 1.9440
Epoch 20/20
5/5 - 0s - 6ms/step - loss: 1.9194
2/2 - 0s - 154ms/step
[CV] END .............................l1_reg=0.01, n_units=5; total time=   2.0s
Epoch 1/20
7/7 - 1s - 160ms/step - loss: 2.5121
Epoch 2/20
7/7 - 0s - 4ms/step - loss: 2.4550
Epoch 3/20
7/7 - 0s - 5ms/step - loss: 2.4114
Epoch 4/20
7/7 - 0s - 5ms/step - loss: 2.3620
Epoch 5/20
7/7 - 0s - 5ms/step - loss: 2.3174
Epoch 6/20
7/7 - 0s - 5ms/step - loss: 2.2700
Epoch 7/20
7/7 - 0s - 5ms/step - loss: 2.2295
Epoch 8/20
7/7 - 0s - 5ms/step - loss: 2.1858
Epoch 9/20
7/7 - 0s - 6ms/step - loss: 2.1441
Epoch 10/20
7/7 - 0s - 6ms/step - loss: 2.1034
Epoch 11/20
7/7 - 0s - 6ms/step - loss: 2.0695
Epoch 12/20
7/7 - 0s - 4ms/step - loss: 2.0330
Epoch 13/20
7/7 - 0s - 5ms/step - loss: 2.0019
Epoch 14/20
7/7 - 0s - 4ms/step - loss: 1.9677
Epoch 15/20
7/7 - 0s - 4ms/step - loss: 1.9345
Epoch 16/20
7/7 - 0s - 5ms/step - loss: 1.9033
Epoch 17/20
7/7 - 0s - 5ms/step - loss: 1.8701
Epoch 18/20
7/7 - 0s - 5ms/step - loss: 1.8453
Epoch 19/20
7/7 - 0s - 5ms/step - loss: 1.8139
Epoch 20/20
7/7 - 0s - 5ms/step - loss: 1.7867
2/2 - 0s - 122ms/step
[CV] END .............................l1_reg=0.01, n_units=5; total time=   2.1s
Epoch 1/20
8/8 - 1s - 132ms/step - loss: 2.2057
Epoch 2/20
8/8 - 0s - 4ms/step - loss: 2.1420
Epoch 3/20
8/8 - 0s - 5ms/step - loss: 2.0880
Epoch 4/20
8/8 - 0s - 4ms/step - loss: 2.0323
Epoch 5/20
8/8 - 0s - 4ms/step - loss: 1.9771
Epoch 6/20
8/8 - 0s - 4ms/step - loss: 1.9279
Epoch 7/20
8/8 - 0s - 5ms/step - loss: 1.8857
Epoch 8/20
8/8 - 0s - 4ms/step - loss: 1.8405
Epoch 9/20
8/8 - 0s - 5ms/step - loss: 1.7964
Epoch 10/20
8/8 - 0s - 4ms/step - loss: 1.7600
Epoch 11/20
8/8 - 0s - 4ms/step - loss: 1.7226
Epoch 12/20
8/8 - 0s - 4ms/step - loss: 1.6904
Epoch 13/20
8/8 - 0s - 4ms/step - loss: 1.6546
Epoch 14/20
8/8 - 0s - 4ms/step - loss: 1.6278
Epoch 15/20
8/8 - 0s - 4ms/step - loss: 1.5909
Epoch 16/20
8/8 - 0s - 5ms/step - loss: 1.5648
Epoch 17/20
8/8 - 0s - 4ms/step - loss: 1.5342
Epoch 18/20
8/8 - 0s - 4ms/step - loss: 1.5098
Epoch 19/20
8/8 - 0s - 7ms/step - loss: 1.4840
Epoch 20/20
8/8 - 0s - 5ms/step - loss: 1.4581
2/2 - 0s - 127ms/step
[CV] END .............................l1_reg=0.01, n_units=5; total time=   2.0s
Epoch 1/20
2/2 - 1s - 547ms/step - loss: 4.5840
Epoch 2/20
2/2 - 0s - 12ms/step - loss: 4.5149
Epoch 3/20
2/2 - 0s - 12ms/step - loss: 4.4567
Epoch 4/20
2/2 - 0s - 13ms/step - loss: 4.3857
Epoch 5/20
2/2 - 0s - 12ms/step - loss: 4.3279
Epoch 6/20
2/2 - 0s - 13ms/step - loss: 4.2732
Epoch 7/20
2/2 - 0s - 13ms/step - loss: 4.2112
Epoch 8/20
2/2 - 0s - 14ms/step - loss: 4.1525
Epoch 9/20
2/2 - 0s - 16ms/step - loss: 4.1043
Epoch 10/20
2/2 - 0s - 20ms/step - loss: 4.0503
Epoch 11/20
2/2 - 0s - 16ms/step - loss: 4.0064
Epoch 12/20
2/2 - 0s - 16ms/step - loss: 3.9517
Epoch 13/20
2/2 - 0s - 12ms/step - loss: 3.9083
Epoch 14/20
2/2 - 0s - 15ms/step - loss: 3.8726
Epoch 15/20
2/2 - 0s - 14ms/step - loss: 3.8217
Epoch 16/20
2/2 - 0s - 13ms/step - loss: 3.7855
Epoch 17/20
2/2 - 0s - 15ms/step - loss: 3.7449
Epoch 18/20
2/2 - 0s - 22ms/step - loss: 3.7099
Epoch 19/20
2/2 - 0s - 19ms/step - loss: 3.6666
Epoch 20/20
2/2 - 0s - 18ms/step - loss: 3.6364
2/2 - 0s - 117ms/step
[CV] END ............................l1_reg=0.01, n_units=10; total time=   1.9s
Epoch 1/20
4/4 - 1s - 266ms/step - loss: 4.0647
Epoch 2/20
4/4 - 0s - 7ms/step - loss: 3.9904
Epoch 3/20
4/4 - 0s - 7ms/step - loss: 3.9137
Epoch 4/20
4/4 - 0s - 8ms/step - loss: 3.8423
Epoch 5/20
4/4 - 0s - 10ms/step - loss: 3.7709
Epoch 6/20
4/4 - 0s - 7ms/step - loss: 3.6988
Epoch 7/20
4/4 - 0s - 12ms/step - loss: 3.6245
Epoch 8/20
4/4 - 0s - 11ms/step - loss: 3.5661
Epoch 9/20
4/4 - 0s - 12ms/step - loss: 3.5011
Epoch 10/20
4/4 - 0s - 11ms/step - loss: 3.4416
Epoch 11/20
4/4 - 0s - 14ms/step - loss: 3.3802
Epoch 12/20
4/4 - 0s - 15ms/step - loss: 3.3300
Epoch 13/20
4/4 - 0s - 11ms/step - loss: 3.2802
Epoch 14/20
4/4 - 0s - 9ms/step - loss: 3.2205
Epoch 15/20
4/4 - 0s - 10ms/step - loss: 3.1707
Epoch 16/20
4/4 - 0s - 9ms/step - loss: 3.1243
Epoch 17/20
4/4 - 0s - 9ms/step - loss: 3.0812
Epoch 18/20
4/4 - 0s - 10ms/step - loss: 3.0369
Epoch 19/20
4/4 - 0s - 8ms/step - loss: 3.0037
Epoch 20/20
4/4 - 0s - 8ms/step - loss: 2.9593
2/2 - 0s - 141ms/step
[CV] END ............................l1_reg=0.01, n_units=10; total time=   2.2s
Epoch 1/20
5/5 - 1s - 247ms/step - loss: 3.5686
Epoch 2/20
5/5 - 0s - 6ms/step - loss: 3.4591
Epoch 3/20
5/5 - 0s - 8ms/step - loss: 3.3596
Epoch 4/20
5/5 - 0s - 6ms/step - loss: 3.2727
Epoch 5/20
5/5 - 0s - 9ms/step - loss: 3.1933
Epoch 6/20
5/5 - 0s - 6ms/step - loss: 3.1080
Epoch 7/20
5/5 - 0s - 8ms/step - loss: 3.0334
Epoch 8/20
5/5 - 0s - 7ms/step - loss: 2.9621
Epoch 9/20
5/5 - 0s - 8ms/step - loss: 2.9032
Epoch 10/20
5/5 - 0s - 6ms/step - loss: 2.8391
Epoch 11/20
5/5 - 0s - 7ms/step - loss: 2.7755
Epoch 12/20
5/5 - 0s - 9ms/step - loss: 2.7182
Epoch 13/20
5/5 - 0s - 7ms/step - loss: 2.6673
Epoch 14/20
5/5 - 0s - 7ms/step - loss: 2.6159
Epoch 15/20
5/5 - 0s - 10ms/step - loss: 2.5625
Epoch 16/20
5/5 - 0s - 9ms/step - loss: 2.5168
Epoch 17/20
5/5 - 0s - 8ms/step - loss: 2.4677
Epoch 18/20
5/5 - 0s - 6ms/step - loss: 2.4269
Epoch 19/20
5/5 - 0s - 6ms/step - loss: 2.3877
Epoch 20/20
5/5 - 0s - 6ms/step - loss: 2.3470
2/2 - 0s - 161ms/step
[CV] END ............................l1_reg=0.01, n_units=10; total time=   2.3s
Epoch 1/20
7/7 - 1s - 156ms/step - loss: 3.6272
Epoch 2/20
7/7 - 0s - 6ms/step - loss: 3.5036
Epoch 3/20
7/7 - 0s - 4ms/step - loss: 3.3673
Epoch 4/20
7/7 - 0s - 4ms/step - loss: 3.2707
Epoch 5/20
7/7 - 0s - 5ms/step - loss: 3.1572
Epoch 6/20
7/7 - 0s - 4ms/step - loss: 3.0629
Epoch 7/20
7/7 - 0s - 5ms/step - loss: 2.9697
Epoch 8/20
7/7 - 0s - 5ms/step - loss: 2.8856
Epoch 9/20
7/7 - 0s - 5ms/step - loss: 2.8010
Epoch 10/20
7/7 - 0s - 4ms/step - loss: 2.7118
Epoch 11/20
7/7 - 0s - 5ms/step - loss: 2.6396
Epoch 12/20
7/7 - 0s - 5ms/step - loss: 2.5661
Epoch 13/20
7/7 - 0s - 4ms/step - loss: 2.5045
Epoch 14/20
7/7 - 0s - 5ms/step - loss: 2.4410
Epoch 15/20
7/7 - 0s - 4ms/step - loss: 2.3788
Epoch 16/20
7/7 - 0s - 5ms/step - loss: 2.3234
Epoch 17/20
7/7 - 0s - 7ms/step - loss: 2.2543
Epoch 18/20
7/7 - 0s - 5ms/step - loss: 2.2045
Epoch 19/20
7/7 - 0s - 6ms/step - loss: 2.1583
Epoch 20/20
7/7 - 0s - 6ms/step - loss: 2.1106
2/2 - 0s - 132ms/step
[CV] END ............................l1_reg=0.01, n_units=10; total time=   2.1s
Epoch 1/20
8/8 - 1s - 134ms/step - loss: 3.1716
Epoch 2/20
8/8 - 0s - 5ms/step - loss: 3.0504
Epoch 3/20
8/8 - 0s - 4ms/step - loss: 2.9201
Epoch 4/20
8/8 - 0s - 5ms/step - loss: 2.8187
Epoch 5/20
8/8 - 0s - 4ms/step - loss: 2.7122
Epoch 6/20
8/8 - 0s - 5ms/step - loss: 2.6160
Epoch 7/20
8/8 - 0s - 4ms/step - loss: 2.5225
Epoch 8/20
8/8 - 0s - 5ms/step - loss: 2.4333
Epoch 9/20
8/8 - 0s - 4ms/step - loss: 2.3534
Epoch 10/20
8/8 - 0s - 5ms/step - loss: 2.2727
Epoch 11/20
8/8 - 0s - 4ms/step - loss: 2.2006
Epoch 12/20
8/8 - 0s - 6ms/step - loss: 2.1275
Epoch 13/20
8/8 - 0s - 4ms/step - loss: 2.0640
Epoch 14/20
8/8 - 0s - 6ms/step - loss: 1.9993
Epoch 15/20
8/8 - 0s - 6ms/step - loss: 1.9386
Epoch 16/20
8/8 - 0s - 11ms/step - loss: 1.8812
Epoch 17/20
8/8 - 0s - 7ms/step - loss: 1.8268
Epoch 18/20
8/8 - 0s - 6ms/step - loss: 1.7790
Epoch 19/20
8/8 - 0s - 5ms/step - loss: 1.7306
Epoch 20/20
8/8 - 0s - 5ms/step - loss: 1.6859
2/2 - 0s - 175ms/step
[CV] END ............................l1_reg=0.01, n_units=10; total time=   2.3s
Epoch 1/20
2/2 - 1s - 714ms/step - loss: 5.4770
Epoch 2/20
2/2 - 0s - 12ms/step - loss: 5.3315
Epoch 3/20
2/2 - 0s - 16ms/step - loss: 5.2006
Epoch 4/20
2/2 - 0s - 12ms/step - loss: 5.0621
Epoch 5/20
2/2 - 0s - 12ms/step - loss: 4.9455
Epoch 6/20
2/2 - 0s - 12ms/step - loss: 4.8489
Epoch 7/20
2/2 - 0s - 13ms/step - loss: 4.7552
Epoch 8/20
2/2 - 0s - 17ms/step - loss: 4.6614
Epoch 9/20
2/2 - 0s - 15ms/step - loss: 4.5740
Epoch 10/20
2/2 - 0s - 15ms/step - loss: 4.4973
Epoch 11/20
2/2 - 0s - 15ms/step - loss: 4.4257
Epoch 12/20
2/2 - 0s - 15ms/step - loss: 4.3422
Epoch 13/20
2/2 - 0s - 18ms/step - loss: 4.2948
Epoch 14/20
2/2 - 0s - 13ms/step - loss: 4.2224
Epoch 15/20
2/2 - 0s - 14ms/step - loss: 4.1618
Epoch 16/20
2/2 - 0s - 14ms/step - loss: 4.1032
Epoch 17/20
2/2 - 0s - 13ms/step - loss: 4.0446
Epoch 18/20
2/2 - 0s - 13ms/step - loss: 4.0005
Epoch 19/20
2/2 - 0s - 13ms/step - loss: 3.9349
Epoch 20/20
2/2 - 0s - 21ms/step - loss: 3.8967
2/2 - 1s - 415ms/step
[CV] END ............................l1_reg=0.01, n_units=20; total time=   2.9s
Epoch 1/20
4/4 - 1s - 268ms/step - loss: 4.8097
Epoch 2/20
4/4 - 0s - 8ms/step - loss: 4.6376
Epoch 3/20
4/4 - 0s - 7ms/step - loss: 4.4685
Epoch 4/20
4/4 - 0s - 6ms/step - loss: 4.3247
Epoch 5/20
4/4 - 0s - 7ms/step - loss: 4.1690
Epoch 6/20
4/4 - 0s - 8ms/step - loss: 4.0336
Epoch 7/20
4/4 - 0s - 7ms/step - loss: 3.8941
Epoch 8/20
4/4 - 0s - 7ms/step - loss: 3.7796
Epoch 9/20
4/4 - 0s - 8ms/step - loss: 3.6765
Epoch 10/20
4/4 - 0s - 8ms/step - loss: 3.5854
Epoch 11/20
4/4 - 0s - 11ms/step - loss: 3.5065
Epoch 12/20
4/4 - 0s - 7ms/step - loss: 3.4416
Epoch 13/20
4/4 - 0s - 8ms/step - loss: 3.3795
Epoch 14/20
4/4 - 0s - 8ms/step - loss: 3.3256
Epoch 15/20
4/4 - 0s - 11ms/step - loss: 3.2648
Epoch 16/20
4/4 - 0s - 9ms/step - loss: 3.2082
Epoch 17/20
4/4 - 0s - 8ms/step - loss: 3.1490
Epoch 18/20
4/4 - 0s - 7ms/step - loss: 3.1033
Epoch 19/20
4/4 - 0s - 8ms/step - loss: 3.0567
Epoch 20/20
4/4 - 0s - 10ms/step - loss: 3.0153
2/2 - 0s - 134ms/step
[CV] END ............................l1_reg=0.01, n_units=20; total time=   2.0s
Epoch 1/20
5/5 - 1s - 220ms/step - loss: 4.2030
Epoch 2/20
5/5 - 0s - 6ms/step - loss: 3.9816
Epoch 3/20
5/5 - 0s - 6ms/step - loss: 3.8412
Epoch 4/20
5/5 - 0s - 5ms/step - loss: 3.6866
Epoch 5/20
5/5 - 0s - 6ms/step - loss: 3.5455
Epoch 6/20
5/5 - 0s - 6ms/step - loss: 3.4443
Epoch 7/20
5/5 - 0s - 6ms/step - loss: 3.3326
Epoch 8/20
5/5 - 0s - 6ms/step - loss: 3.2353
Epoch 9/20
5/5 - 0s - 9ms/step - loss: 3.1331
Epoch 10/20
5/5 - 0s - 8ms/step - loss: 3.0424
Epoch 11/20
5/5 - 0s - 6ms/step - loss: 2.9580
Epoch 12/20
5/5 - 0s - 6ms/step - loss: 2.8735
Epoch 13/20
5/5 - 0s - 6ms/step - loss: 2.8062
Epoch 14/20
5/5 - 0s - 8ms/step - loss: 2.7174
Epoch 15/20
5/5 - 0s - 7ms/step - loss: 2.6521
Epoch 16/20
5/5 - 0s - 8ms/step - loss: 2.5867
Epoch 17/20
5/5 - 0s - 7ms/step - loss: 2.5210
Epoch 18/20
5/5 - 0s - 8ms/step - loss: 2.4576
Epoch 19/20
5/5 - 0s - 7ms/step - loss: 2.4040
Epoch 20/20
5/5 - 0s - 9ms/step - loss: 2.3462
2/2 - 0s - 123ms/step
[CV] END ............................l1_reg=0.01, n_units=20; total time=   2.0s
Epoch 1/20
7/7 - 1s - 159ms/step - loss: 4.2673
Epoch 2/20
7/7 - 0s - 4ms/step - loss: 4.0199
Epoch 3/20
7/7 - 0s - 5ms/step - loss: 3.8231
Epoch 4/20
7/7 - 0s - 4ms/step - loss: 3.6529
Epoch 5/20
7/7 - 0s - 6ms/step - loss: 3.5157
Epoch 6/20
7/7 - 0s - 4ms/step - loss: 3.3698
Epoch 7/20
7/7 - 0s - 5ms/step - loss: 3.2315
Epoch 8/20
7/7 - 0s - 6ms/step - loss: 3.1053
Epoch 9/20
7/7 - 0s - 5ms/step - loss: 2.9881
Epoch 10/20
7/7 - 0s - 5ms/step - loss: 2.8713
Epoch 11/20
7/7 - 0s - 7ms/step - loss: 2.7749
Epoch 12/20
7/7 - 0s - 6ms/step - loss: 2.6715
Epoch 13/20
7/7 - 0s - 7ms/step - loss: 2.5742
Epoch 14/20
7/7 - 0s - 5ms/step - loss: 2.4934
Epoch 15/20
7/7 - 0s - 5ms/step - loss: 2.4162
Epoch 16/20
7/7 - 0s - 5ms/step - loss: 2.3432
Epoch 17/20
7/7 - 0s - 6ms/step - loss: 2.2758
Epoch 18/20
7/7 - 0s - 5ms/step - loss: 2.2057
Epoch 19/20
7/7 - 0s - 6ms/step - loss: 2.1379
Epoch 20/20
7/7 - 0s - 6ms/step - loss: 2.0887
2/2 - 0s - 121ms/step
[CV] END ............................l1_reg=0.01, n_units=20; total time=   2.1s
Epoch 1/20
8/8 - 1s - 137ms/step - loss: 3.7279
Epoch 2/20
8/8 - 0s - 5ms/step - loss: 3.4856
Epoch 3/20
8/8 - 0s - 5ms/step - loss: 3.2749
Epoch 4/20
8/8 - 0s - 4ms/step - loss: 3.0839
Epoch 5/20
8/8 - 0s - 4ms/step - loss: 2.9199
Epoch 6/20
8/8 - 0s - 5ms/step - loss: 2.7773
Epoch 7/20
8/8 - 0s - 4ms/step - loss: 2.6471
Epoch 8/20
8/8 - 0s - 5ms/step - loss: 2.5217
Epoch 9/20
8/8 - 0s - 5ms/step - loss: 2.3945
Epoch 10/20
8/8 - 0s - 5ms/step - loss: 2.2954
Epoch 11/20
8/8 - 0s - 5ms/step - loss: 2.1964
Epoch 12/20
8/8 - 0s - 4ms/step - loss: 2.1033
Epoch 13/20
8/8 - 0s - 4ms/step - loss: 2.0153
Epoch 14/20
8/8 - 0s - 4ms/step - loss: 1.9417
Epoch 15/20
8/8 - 0s - 5ms/step - loss: 1.8688
Epoch 16/20
8/8 - 0s - 6ms/step - loss: 1.8092
Epoch 17/20
8/8 - 0s - 5ms/step - loss: 1.7448
Epoch 18/20
8/8 - 0s - 5ms/step - loss: 1.6970
Epoch 19/20
8/8 - 0s - 7ms/step - loss: 1.6399
Epoch 20/20
8/8 - 0s - 6ms/step - loss: 1.5967
2/2 - 0s - 121ms/step
[CV] END ............................l1_reg=0.01, n_units=20; total time=   2.1s
Epoch 1/20
2/2 - 1s - 522ms/step - loss: 3.5905
Epoch 2/20
2/2 - 0s - 12ms/step - loss: 3.5674
Epoch 3/20
2/2 - 0s - 13ms/step - loss: 3.5387
Epoch 4/20
2/2 - 0s - 12ms/step - loss: 3.5178
Epoch 5/20
2/2 - 0s - 13ms/step - loss: 3.4919
Epoch 6/20
2/2 - 0s - 14ms/step - loss: 3.4694
Epoch 7/20
2/2 - 0s - 13ms/step - loss: 3.4458
Epoch 8/20
2/2 - 0s - 18ms/step - loss: 3.4230
Epoch 9/20
2/2 - 0s - 20ms/step - loss: 3.3997
Epoch 10/20
2/2 - 0s - 16ms/step - loss: 3.3773
Epoch 11/20
2/2 - 0s - 15ms/step - loss: 3.3606
Epoch 12/20
2/2 - 0s - 13ms/step - loss: 3.3353
Epoch 13/20
2/2 - 0s - 17ms/step - loss: 3.3154
Epoch 14/20
2/2 - 0s - 20ms/step - loss: 3.2938
Epoch 15/20
2/2 - 0s - 13ms/step - loss: 3.2743
Epoch 16/20
2/2 - 0s - 16ms/step - loss: 3.2518
Epoch 17/20
2/2 - 0s - 21ms/step - loss: 3.2326
Epoch 18/20
2/2 - 0s - 18ms/step - loss: 3.2139
Epoch 19/20
2/2 - 0s - 16ms/step - loss: 3.1932
Epoch 20/20
2/2 - 0s - 13ms/step - loss: 3.1734
2/2 - 0s - 134ms/step
[CV] END ..............................l1_reg=0.1, n_units=5; total time=   1.9s
Epoch 1/20
4/4 - 1s - 280ms/step - loss: 3.3037
Epoch 2/20
4/4 - 0s - 6ms/step - loss: 3.2650
Epoch 3/20
4/4 - 0s - 9ms/step - loss: 3.2310
Epoch 4/20
4/4 - 0s - 7ms/step - loss: 3.2034
Epoch 5/20
4/4 - 0s - 6ms/step - loss: 3.1661
Epoch 6/20
4/4 - 0s - 7ms/step - loss: 3.1372
Epoch 7/20
4/4 - 0s - 8ms/step - loss: 3.1071
Epoch 8/20
4/4 - 0s - 8ms/step - loss: 3.0788
Epoch 9/20
4/4 - 0s - 9ms/step - loss: 3.0541
Epoch 10/20
4/4 - 0s - 11ms/step - loss: 3.0279
Epoch 11/20
4/4 - 0s - 10ms/step - loss: 3.0067
Epoch 12/20
4/4 - 0s - 9ms/step - loss: 2.9781
Epoch 13/20
4/4 - 0s - 8ms/step - loss: 2.9560
Epoch 14/20
4/4 - 0s - 7ms/step - loss: 2.9316
Epoch 15/20
4/4 - 0s - 7ms/step - loss: 2.9128
Epoch 16/20
4/4 - 0s - 11ms/step - loss: 2.8908
Epoch 17/20
4/4 - 0s - 10ms/step - loss: 2.8665
Epoch 18/20
4/4 - 0s - 9ms/step - loss: 2.8432
Epoch 19/20
4/4 - 0s - 10ms/step - loss: 2.8217
Epoch 20/20
4/4 - 0s - 9ms/step - loss: 2.8035
2/2 - 0s - 125ms/step
[CV] END ..............................l1_reg=0.1, n_units=5; total time=   2.1s
Epoch 1/20
5/5 - 1s - 215ms/step - loss: 2.8999
Epoch 2/20
5/5 - 0s - 6ms/step - loss: 2.8588
Epoch 3/20
5/5 - 0s - 7ms/step - loss: 2.8159
Epoch 4/20
5/5 - 0s - 6ms/step - loss: 2.7730
Epoch 5/20
5/5 - 0s - 6ms/step - loss: 2.7383
Epoch 6/20
5/5 - 0s - 6ms/step - loss: 2.6998
Epoch 7/20
5/5 - 0s - 7ms/step - loss: 2.6625
Epoch 8/20
5/5 - 0s - 6ms/step - loss: 2.6253
Epoch 9/20
5/5 - 0s - 6ms/step - loss: 2.5903
Epoch 10/20
5/5 - 0s - 7ms/step - loss: 2.5548
Epoch 11/20
5/5 - 0s - 6ms/step - loss: 2.5214
Epoch 12/20
5/5 - 0s - 8ms/step - loss: 2.4882
Epoch 13/20
5/5 - 0s - 6ms/step - loss: 2.4581
Epoch 14/20
5/5 - 0s - 7ms/step - loss: 2.4300
Epoch 15/20
5/5 - 0s - 6ms/step - loss: 2.4001
Epoch 16/20
5/5 - 0s - 5ms/step - loss: 2.3667
Epoch 17/20
5/5 - 0s - 7ms/step - loss: 2.3406
Epoch 18/20
5/5 - 0s - 10ms/step - loss: 2.3129
Epoch 19/20
5/5 - 0s - 6ms/step - loss: 2.2862
Epoch 20/20
5/5 - 0s - 7ms/step - loss: 2.2592
2/2 - 0s - 126ms/step
[CV] END ..............................l1_reg=0.1, n_units=5; total time=   2.0s
Epoch 1/20
7/7 - 1s - 154ms/step - loss: 2.9300
Epoch 2/20
7/7 - 0s - 4ms/step - loss: 2.8659
Epoch 3/20
7/7 - 0s - 5ms/step - loss: 2.8000
Epoch 4/20
7/7 - 0s - 4ms/step - loss: 2.7474
Epoch 5/20
7/7 - 0s - 4ms/step - loss: 2.6957
Epoch 6/20
7/7 - 0s - 4ms/step - loss: 2.6486
Epoch 7/20
7/7 - 0s - 6ms/step - loss: 2.5974
Epoch 8/20
7/7 - 0s - 5ms/step - loss: 2.5499
Epoch 9/20
7/7 - 0s - 5ms/step - loss: 2.4965
Epoch 10/20
7/7 - 0s - 5ms/step - loss: 2.4556
Epoch 11/20
7/7 - 0s - 4ms/step - loss: 2.4175
Epoch 12/20
7/7 - 0s - 5ms/step - loss: 2.3808
Epoch 13/20
7/7 - 0s - 5ms/step - loss: 2.3421
Epoch 14/20
7/7 - 0s - 5ms/step - loss: 2.3046
Epoch 15/20
7/7 - 0s - 8ms/step - loss: 2.2699
Epoch 16/20
7/7 - 0s - 5ms/step - loss: 2.2377
Epoch 17/20
7/7 - 0s - 6ms/step - loss: 2.2080
Epoch 18/20
7/7 - 0s - 5ms/step - loss: 2.1787
Epoch 19/20
7/7 - 0s - 4ms/step - loss: 2.1482
Epoch 20/20
7/7 - 0s - 5ms/step - loss: 2.1218
2/2 - 0s - 127ms/step
[CV] END ..............................l1_reg=0.1, n_units=5; total time=   2.0s
Epoch 1/20
8/8 - 1s - 137ms/step - loss: 2.6300
Epoch 2/20
8/8 - 0s - 4ms/step - loss: 2.5612
Epoch 3/20
8/8 - 0s - 5ms/step - loss: 2.4967
Epoch 4/20
8/8 - 0s - 4ms/step - loss: 2.4359
Epoch 5/20
8/8 - 0s - 4ms/step - loss: 2.3743
Epoch 6/20
8/8 - 0s - 4ms/step - loss: 2.3226
Epoch 7/20
8/8 - 0s - 4ms/step - loss: 2.2642
Epoch 8/20
8/8 - 0s - 4ms/step - loss: 2.2100
Epoch 9/20
8/8 - 0s - 5ms/step - loss: 2.1651
Epoch 10/20
8/8 - 0s - 5ms/step - loss: 2.1139
Epoch 11/20
8/8 - 0s - 5ms/step - loss: 2.0711
Epoch 12/20
8/8 - 0s - 6ms/step - loss: 2.0278
Epoch 13/20
8/8 - 0s - 4ms/step - loss: 1.9873
Epoch 14/20
8/8 - 0s - 4ms/step - loss: 1.9486
Epoch 15/20
8/8 - 0s - 4ms/step - loss: 1.9111
Epoch 16/20
8/8 - 0s - 4ms/step - loss: 1.8740
Epoch 17/20
8/8 - 0s - 4ms/step - loss: 1.8377
Epoch 18/20
8/8 - 0s - 4ms/step - loss: 1.8100
Epoch 19/20
8/8 - 0s - 5ms/step - loss: 1.7776
Epoch 20/20
8/8 - 0s - 6ms/step - loss: 1.7466
2/2 - 0s - 128ms/step
[CV] END ..............................l1_reg=0.1, n_units=5; total time=   2.1s
Epoch 1/20
2/2 - 1s - 579ms/step - loss: 5.2646
Epoch 2/20
2/2 - 0s - 12ms/step - loss: 5.1938
Epoch 3/20
2/2 - 0s - 13ms/step - loss: 5.1195
Epoch 4/20
2/2 - 0s - 13ms/step - loss: 5.0519
Epoch 5/20
2/2 - 0s - 13ms/step - loss: 4.9943
Epoch 6/20
2/2 - 0s - 13ms/step - loss: 4.9299
Epoch 7/20
2/2 - 0s - 13ms/step - loss: 4.8697
Epoch 8/20
2/2 - 0s - 13ms/step - loss: 4.8141
Epoch 9/20
2/2 - 0s - 12ms/step - loss: 4.7546
Epoch 10/20
2/2 - 0s - 14ms/step - loss: 4.7020
Epoch 11/20
2/2 - 0s - 16ms/step - loss: 4.6483
Epoch 12/20
2/2 - 0s - 19ms/step - loss: 4.5955
Epoch 13/20
2/2 - 0s - 18ms/step - loss: 4.5460
Epoch 14/20
2/2 - 0s - 22ms/step - loss: 4.4972
Epoch 15/20
2/2 - 0s - 20ms/step - loss: 4.4502
Epoch 16/20
2/2 - 0s - 14ms/step - loss: 4.4039
Epoch 17/20
2/2 - 0s - 26ms/step - loss: 4.3636
Epoch 18/20
2/2 - 0s - 62ms/step - loss: 4.3182
Epoch 19/20
2/2 - 0s - 31ms/step - loss: 4.2749
Epoch 20/20
2/2 - 0s - 18ms/step - loss: 4.2350
2/2 - 0s - 202ms/step
[CV] END .............................l1_reg=0.1, n_units=10; total time=   2.4s
Epoch 1/20
4/4 - 1s - 320ms/step - loss: 4.7710
Epoch 2/20
4/4 - 0s - 9ms/step - loss: 4.6608
Epoch 3/20
4/4 - 0s - 8ms/step - loss: 4.5732
Epoch 4/20
4/4 - 0s - 8ms/step - loss: 4.4795
Epoch 5/20
4/4 - 0s - 8ms/step - loss: 4.3982
Epoch 6/20
4/4 - 0s - 8ms/step - loss: 4.3112
Epoch 7/20
4/4 - 0s - 9ms/step - loss: 4.2425
Epoch 8/20
4/4 - 0s - 10ms/step - loss: 4.1666
Epoch 9/20
4/4 - 0s - 10ms/step - loss: 4.0989
Epoch 10/20
4/4 - 0s - 9ms/step - loss: 4.0385
Epoch 11/20
4/4 - 0s - 8ms/step - loss: 3.9718
Epoch 12/20
4/4 - 0s - 10ms/step - loss: 3.9150
Epoch 13/20
4/4 - 0s - 8ms/step - loss: 3.8615
Epoch 14/20
4/4 - 0s - 8ms/step - loss: 3.8066
Epoch 15/20
4/4 - 0s - 9ms/step - loss: 3.7562
Epoch 16/20
4/4 - 0s - 10ms/step - loss: 3.7117
Epoch 17/20
4/4 - 0s - 11ms/step - loss: 3.6606
Epoch 18/20
4/4 - 0s - 7ms/step - loss: 3.6186
Epoch 19/20
4/4 - 0s - 8ms/step - loss: 3.5709
Epoch 20/20
4/4 - 0s - 9ms/step - loss: 3.5191
2/2 - 0s - 178ms/step
[CV] END .............................l1_reg=0.1, n_units=10; total time=   2.4s
Epoch 1/20
5/5 - 1s - 291ms/step - loss: 4.2335
Epoch 2/20
5/5 - 0s - 8ms/step - loss: 4.1198
Epoch 3/20
5/5 - 0s - 6ms/step - loss: 4.0105
Epoch 4/20
5/5 - 0s - 7ms/step - loss: 3.9185
Epoch 5/20
5/5 - 0s - 9ms/step - loss: 3.8285
Epoch 6/20
5/5 - 0s - 8ms/step - loss: 3.7359
Epoch 7/20
5/5 - 0s - 6ms/step - loss: 3.6628
Epoch 8/20
5/5 - 0s - 8ms/step - loss: 3.5790
Epoch 9/20
5/5 - 0s - 6ms/step - loss: 3.5081
Epoch 10/20
5/5 - 0s - 6ms/step - loss: 3.4302
Epoch 11/20
5/5 - 0s - 9ms/step - loss: 3.3726
Epoch 12/20
5/5 - 0s - 7ms/step - loss: 3.3010
Epoch 13/20
5/5 - 0s - 6ms/step - loss: 3.2382
Epoch 14/20
5/5 - 0s - 8ms/step - loss: 3.1832
Epoch 15/20
5/5 - 0s - 6ms/step - loss: 3.1255
Epoch 16/20
5/5 - 0s - 7ms/step - loss: 3.0717
Epoch 17/20
5/5 - 0s - 7ms/step - loss: 3.0136
Epoch 18/20
5/5 - 0s - 8ms/step - loss: 2.9665
Epoch 19/20
5/5 - 0s - 9ms/step - loss: 2.9161
Epoch 20/20
5/5 - 0s - 6ms/step - loss: 2.8718
2/2 - 0s - 125ms/step
[CV] END .............................l1_reg=0.1, n_units=10; total time=   2.5s
Epoch 1/20
7/7 - 1s - 188ms/step - loss: 4.2751
Epoch 2/20
7/7 - 0s - 5ms/step - loss: 4.1411
Epoch 3/20
7/7 - 0s - 4ms/step - loss: 3.9961
Epoch 4/20
7/7 - 0s - 4ms/step - loss: 3.8881
Epoch 5/20
7/7 - 0s - 5ms/step - loss: 3.7801
Epoch 6/20
7/7 - 0s - 5ms/step - loss: 3.6660
Epoch 7/20
7/7 - 0s - 5ms/step - loss: 3.5677
Epoch 8/20
7/7 - 0s - 5ms/step - loss: 3.4751
Epoch 9/20
7/7 - 0s - 6ms/step - loss: 3.3751
Epoch 10/20
7/7 - 0s - 5ms/step - loss: 3.2826
Epoch 11/20
7/7 - 0s - 6ms/step - loss: 3.1904
Epoch 12/20
7/7 - 0s - 6ms/step - loss: 3.1093
Epoch 13/20
7/7 - 0s - 5ms/step - loss: 3.0416
Epoch 14/20
7/7 - 0s - 4ms/step - loss: 2.9727
Epoch 15/20
7/7 - 0s - 7ms/step - loss: 2.9089
Epoch 16/20
7/7 - 0s - 4ms/step - loss: 2.8450
Epoch 17/20
7/7 - 0s - 5ms/step - loss: 2.7842
Epoch 18/20
7/7 - 0s - 5ms/step - loss: 2.7268
Epoch 19/20
7/7 - 0s - 5ms/step - loss: 2.6707
Epoch 20/20
7/7 - 0s - 4ms/step - loss: 2.6143
2/2 - 0s - 122ms/step
[CV] END .............................l1_reg=0.1, n_units=10; total time=   2.7s
Epoch 1/20
8/8 - 1s - 132ms/step - loss: 3.8483
Epoch 2/20
8/8 - 0s - 4ms/step - loss: 3.7007
Epoch 3/20
8/8 - 0s - 4ms/step - loss: 3.5608
Epoch 4/20
8/8 - 0s - 5ms/step - loss: 3.4366
Epoch 5/20
8/8 - 0s - 4ms/step - loss: 3.3247
Epoch 6/20
8/8 - 0s - 4ms/step - loss: 3.2107
Epoch 7/20
8/8 - 0s - 4ms/step - loss: 3.1026
Epoch 8/20
8/8 - 0s - 5ms/step - loss: 3.0096
Epoch 9/20
8/8 - 0s - 4ms/step - loss: 2.9114
Epoch 10/20
8/8 - 0s - 4ms/step - loss: 2.8275
Epoch 11/20
8/8 - 0s - 5ms/step - loss: 2.7442
Epoch 12/20
8/8 - 0s - 4ms/step - loss: 2.6626
Epoch 13/20
8/8 - 0s - 5ms/step - loss: 2.5847
Epoch 14/20
8/8 - 0s - 4ms/step - loss: 2.5113
Epoch 15/20
8/8 - 0s - 4ms/step - loss: 2.4432
Epoch 16/20
8/8 - 0s - 6ms/step - loss: 2.3750
Epoch 17/20
8/8 - 0s - 5ms/step - loss: 2.3112
Epoch 18/20
8/8 - 0s - 6ms/step - loss: 2.2537
Epoch 19/20
8/8 - 0s - 5ms/step - loss: 2.1958
Epoch 20/20
8/8 - 0s - 6ms/step - loss: 2.1399
2/2 - 0s - 123ms/step
[CV] END .............................l1_reg=0.1, n_units=10; total time=   2.0s
Epoch 1/20
2/2 - 1s - 564ms/step - loss: 6.4387
Epoch 2/20
2/2 - 0s - 12ms/step - loss: 6.2809
Epoch 3/20
2/2 - 0s - 15ms/step - loss: 6.1378
Epoch 4/20
2/2 - 0s - 13ms/step - loss: 5.9955
Epoch 5/20
2/2 - 0s - 15ms/step - loss: 5.8933
Epoch 6/20
2/2 - 0s - 12ms/step - loss: 5.7696
Epoch 7/20
2/2 - 0s - 14ms/step - loss: 5.6550
Epoch 8/20
2/2 - 0s - 12ms/step - loss: 5.5466
Epoch 9/20
2/2 - 0s - 15ms/step - loss: 5.4634
Epoch 10/20
2/2 - 0s - 16ms/step - loss: 5.3830
Epoch 11/20
2/2 - 0s - 12ms/step - loss: 5.3024
Epoch 12/20
2/2 - 0s - 11ms/step - loss: 5.2129
Epoch 13/20
2/2 - 0s - 15ms/step - loss: 5.1527
Epoch 14/20
2/2 - 0s - 17ms/step - loss: 5.0665
Epoch 15/20
2/2 - 0s - 21ms/step - loss: 4.9992
Epoch 16/20
2/2 - 0s - 19ms/step - loss: 4.9474
Epoch 17/20
2/2 - 0s - 20ms/step - loss: 4.8893
Epoch 18/20
2/2 - 0s - 13ms/step - loss: 4.8268
Epoch 19/20
2/2 - 0s - 12ms/step - loss: 4.7672
Epoch 20/20
2/2 - 0s - 19ms/step - loss: 4.7155
2/2 - 0s - 129ms/step
[CV] END .............................l1_reg=0.1, n_units=20; total time=   2.0s
Epoch 1/20
4/4 - 1s - 274ms/step - loss: 5.7656
Epoch 2/20
4/4 - 0s - 9ms/step - loss: 5.5288
Epoch 3/20
4/4 - 0s - 6ms/step - loss: 5.3478
Epoch 4/20
4/4 - 0s - 7ms/step - loss: 5.1938
Epoch 5/20
4/4 - 0s - 7ms/step - loss: 5.0645
Epoch 6/20
4/4 - 0s - 8ms/step - loss: 4.9296
Epoch 7/20
4/4 - 0s - 8ms/step - loss: 4.7871
Epoch 8/20
4/4 - 0s - 9ms/step - loss: 4.7005
Epoch 9/20
4/4 - 0s - 8ms/step - loss: 4.5819
Epoch 10/20
4/4 - 0s - 8ms/step - loss: 4.4994
Epoch 11/20
4/4 - 0s - 10ms/step - loss: 4.4020
Epoch 12/20
4/4 - 0s - 10ms/step - loss: 4.3286
Epoch 13/20
4/4 - 0s - 11ms/step - loss: 4.2544
Epoch 14/20
4/4 - 0s - 10ms/step - loss: 4.1809
Epoch 15/20
4/4 - 0s - 9ms/step - loss: 4.0979
Epoch 16/20
4/4 - 0s - 10ms/step - loss: 4.0239
Epoch 17/20
4/4 - 0s - 8ms/step - loss: 3.9463
Epoch 18/20
4/4 - 0s - 7ms/step - loss: 3.8795
Epoch 19/20
4/4 - 0s - 9ms/step - loss: 3.8144
Epoch 20/20
4/4 - 0s - 13ms/step - loss: 3.7624
2/2 - 0s - 121ms/step
[CV] END .............................l1_reg=0.1, n_units=20; total time=   2.0s
Epoch 1/20
5/5 - 1s - 220ms/step - loss: 5.1832
Epoch 2/20
5/5 - 0s - 6ms/step - loss: 4.9453
Epoch 3/20
5/5 - 0s - 7ms/step - loss: 4.7434
Epoch 4/20
5/5 - 0s - 5ms/step - loss: 4.5855
Epoch 5/20
5/5 - 0s - 6ms/step - loss: 4.4112
Epoch 6/20
5/5 - 0s - 6ms/step - loss: 4.2708
Epoch 7/20
5/5 - 0s - 6ms/step - loss: 4.1575
Epoch 8/20
5/5 - 0s - 6ms/step - loss: 4.0270
Epoch 9/20
5/5 - 0s - 6ms/step - loss: 3.9236
Epoch 10/20
5/5 - 0s - 7ms/step - loss: 3.8010
Epoch 11/20
5/5 - 0s - 10ms/step - loss: 3.7086
Epoch 12/20
5/5 - 0s - 8ms/step - loss: 3.6100
Epoch 13/20
5/5 - 0s - 6ms/step - loss: 3.5240
Epoch 14/20
5/5 - 0s - 7ms/step - loss: 3.4338
Epoch 15/20
5/5 - 0s - 7ms/step - loss: 3.3539
Epoch 16/20
5/5 - 0s - 7ms/step - loss: 3.2704
Epoch 17/20
5/5 - 0s - 6ms/step - loss: 3.2028
Epoch 18/20
5/5 - 0s - 6ms/step - loss: 3.1292
Epoch 19/20
5/5 - 0s - 6ms/step - loss: 3.0575
Epoch 20/20
5/5 - 0s - 6ms/step - loss: 2.9982
2/2 - 0s - 127ms/step
[CV] END .............................l1_reg=0.1, n_units=20; total time=   2.0s
Epoch 1/20
7/7 - 1s - 155ms/step - loss: 5.2528
Epoch 2/20
7/7 - 0s - 4ms/step - loss: 4.9700
Epoch 3/20
7/7 - 0s - 5ms/step - loss: 4.7295
Epoch 4/20
7/7 - 0s - 4ms/step - loss: 4.5126
Epoch 5/20
7/7 - 0s - 4ms/step - loss: 4.2922
Epoch 6/20
7/7 - 0s - 4ms/step - loss: 4.1178
Epoch 7/20
7/7 - 0s - 5ms/step - loss: 3.9596
Epoch 8/20
7/7 - 0s - 5ms/step - loss: 3.8087
Epoch 9/20
7/7 - 0s - 6ms/step - loss: 3.6856
Epoch 10/20
7/7 - 0s - 6ms/step - loss: 3.5651
Epoch 11/20
7/7 - 0s - 5ms/step - loss: 3.4486
Epoch 12/20
7/7 - 0s - 5ms/step - loss: 3.3407
Epoch 13/20
7/7 - 0s - 4ms/step - loss: 3.2254
Epoch 14/20
7/7 - 0s - 5ms/step - loss: 3.1368
Epoch 15/20
7/7 - 0s - 5ms/step - loss: 3.0408
Epoch 16/20
7/7 - 0s - 5ms/step - loss: 2.9516
Epoch 17/20
7/7 - 0s - 5ms/step - loss: 2.8642
Epoch 18/20
7/7 - 0s - 4ms/step - loss: 2.7839
Epoch 19/20
7/7 - 0s - 7ms/step - loss: 2.7040
Epoch 20/20
7/7 - 0s - 7ms/step - loss: 2.6534
2/2 - 0s - 135ms/step
[CV] END .............................l1_reg=0.1, n_units=20; total time=   2.1s
Epoch 1/20
8/8 - 1s - 159ms/step - loss: 4.6756
Epoch 2/20
8/8 - 0s - 4ms/step - loss: 4.4203
Epoch 3/20
8/8 - 0s - 4ms/step - loss: 4.1732
Epoch 4/20
8/8 - 0s - 5ms/step - loss: 3.9764
Epoch 5/20
8/8 - 0s - 4ms/step - loss: 3.7690
Epoch 6/20
8/8 - 0s - 4ms/step - loss: 3.6027
Epoch 7/20
8/8 - 0s - 4ms/step - loss: 3.4377
Epoch 8/20
8/8 - 0s - 5ms/step - loss: 3.2797
Epoch 9/20
8/8 - 0s - 6ms/step - loss: 3.1393
Epoch 10/20
8/8 - 0s - 5ms/step - loss: 3.0155
Epoch 11/20
8/8 - 0s - 5ms/step - loss: 2.9021
Epoch 12/20
8/8 - 0s - 5ms/step - loss: 2.7856
Epoch 13/20
8/8 - 0s - 4ms/step - loss: 2.6844
Epoch 14/20
8/8 - 0s - 5ms/step - loss: 2.5812
Epoch 15/20
8/8 - 0s - 5ms/step - loss: 2.4984
Epoch 16/20
8/8 - 0s - 5ms/step - loss: 2.4161
Epoch 17/20
8/8 - 0s - 6ms/step - loss: 2.3403
Epoch 18/20
8/8 - 0s - 6ms/step - loss: 2.2691
Epoch 19/20
8/8 - 0s - 6ms/step - loss: 2.2131
Epoch 20/20
8/8 - 0s - 5ms/step - loss: 2.1482
2/2 - 0s - 164ms/step
[CV] END .............................l1_reg=0.1, n_units=20; total time=   2.4s
Epoch 1/20
10/10 - 1s - 122ms/step - loss: 2.6518
Epoch 2/20
10/10 - 0s - 3ms/step - loss: 2.5667
Epoch 3/20
10/10 - 0s - 5ms/step - loss: 2.4868
Epoch 4/20
10/10 - 0s - 4ms/step - loss: 2.4183
Epoch 5/20
10/10 - 0s - 3ms/step - loss: 2.3571
Epoch 6/20
10/10 - 0s - 4ms/step - loss: 2.2908
Epoch 7/20
10/10 - 0s - 5ms/step - loss: 2.2339
Epoch 8/20
10/10 - 0s - 4ms/step - loss: 2.1759
Epoch 9/20
10/10 - 0s - 4ms/step - loss: 2.1224
Epoch 10/20
10/10 - 0s - 6ms/step - loss: 2.0699
Epoch 11/20
10/10 - 0s - 4ms/step - loss: 2.0135
Epoch 12/20
10/10 - 0s - 4ms/step - loss: 1.9649
Epoch 13/20
10/10 - 0s - 4ms/step - loss: 1.9204
Epoch 14/20
10/10 - 0s - 4ms/step - loss: 1.8777
Epoch 15/20
10/10 - 0s - 4ms/step - loss: 1.8370
Epoch 16/20
10/10 - 0s - 4ms/step - loss: 1.7995
Epoch 17/20
10/10 - 0s - 4ms/step - loss: 1.7568
Epoch 18/20
10/10 - 0s - 3ms/step - loss: 1.7223
Epoch 19/20
10/10 - 0s - 4ms/step - loss: 1.6867
Epoch 20/20
10/10 - 0s - 5ms/step - loss: 1.6545
Best: -0.781835 using {'l1_reg': 0.1, 'n_units': 5}
nan (nan) with {'l1_reg': 0, 'n_units': 5}
nan (nan) with {'l1_reg': 0, 'n_units': 10}
nan (nan) with {'l1_reg': 0, 'n_units': 20}
-0.803686 (0.349312) with {'l1_reg': 0.001, 'n_units': 5}
-1.331102 (0.571841) with {'l1_reg': 0.001, 'n_units': 10}
-1.398264 (0.776292) with {'l1_reg': 0.001, 'n_units': 20}
-0.798648 (0.347642) with {'l1_reg': 0.01, 'n_units': 5}
-1.296671 (0.576671) with {'l1_reg': 0.01, 'n_units': 10}
-1.337369 (0.684053) with {'l1_reg': 0.01, 'n_units': 20}
-0.781835 (0.322991) with {'l1_reg': 0.1, 'n_units': 5}
-1.280223 (0.551077) with {'l1_reg': 0.1, 'n_units': 10}
-1.337836 (0.776335) with {'l1_reg': 0.1, 'n_units': 20}
Performing cross-validation. Model: gru
Fitting 5 folds for each of 12 candidates, totalling 60 fits
Epoch 1/20
[CV] END ................................l1_reg=0, n_units=5; total time=   0.3s
Epoch 1/20
[CV] END ................................l1_reg=0, n_units=5; total time=   0.2s
Epoch 1/20
[CV] END ................................l1_reg=0, n_units=5; total time=   0.3s
Epoch 1/20
[CV] END ................................l1_reg=0, n_units=5; total time=   0.2s
Epoch 1/20
[CV] END ................................l1_reg=0, n_units=5; total time=   0.2s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=10; total time=   0.2s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=10; total time=   0.2s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=10; total time=   0.2s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=10; total time=   0.2s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=10; total time=   0.2s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=20; total time=   0.2s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=20; total time=   0.2s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=20; total time=   0.2s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=20; total time=   0.2s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=20; total time=   0.2s
Epoch 1/20
2/2 - 3s - 1s/step - loss: 2.5250
Epoch 2/20
2/2 - 0s - 14ms/step - loss: 2.4932
Epoch 3/20
2/2 - 0s - 14ms/step - loss: 2.4553
Epoch 4/20
2/2 - 0s - 38ms/step - loss: 2.4280
Epoch 5/20
2/2 - 0s - 13ms/step - loss: 2.4008
Epoch 6/20
2/2 - 0s - 13ms/step - loss: 2.3648
Epoch 7/20
2/2 - 0s - 15ms/step - loss: 2.3416
Epoch 8/20
2/2 - 0s - 16ms/step - loss: 2.3116
Epoch 9/20
2/2 - 0s - 14ms/step - loss: 2.2864
Epoch 10/20
2/2 - 0s - 22ms/step - loss: 2.2559
Epoch 11/20
2/2 - 0s - 17ms/step - loss: 2.2322
Epoch 12/20
2/2 - 0s - 22ms/step - loss: 2.2051
Epoch 13/20
2/2 - 0s - 21ms/step - loss: 2.1837
Epoch 14/20
2/2 - 0s - 16ms/step - loss: 2.1575
Epoch 15/20
2/2 - 0s - 15ms/step - loss: 2.1298
Epoch 16/20
2/2 - 0s - 13ms/step - loss: 2.1085
Epoch 17/20
2/2 - 0s - 18ms/step - loss: 2.0895
Epoch 18/20
2/2 - 0s - 21ms/step - loss: 2.0651
Epoch 19/20
2/2 - 0s - 18ms/step - loss: 2.0414
Epoch 20/20
2/2 - 0s - 16ms/step - loss: 2.0201
2/2 - 1s - 496ms/step
[CV] END ............................l1_reg=0.001, n_units=5; total time=   4.5s
Epoch 1/20
4/4 - 3s - 725ms/step - loss: 2.5563
Epoch 2/20
4/4 - 0s - 7ms/step - loss: 2.4899
Epoch 3/20
4/4 - 0s - 9ms/step - loss: 2.4285
Epoch 4/20
4/4 - 0s - 8ms/step - loss: 2.3751
Epoch 5/20
4/4 - 0s - 8ms/step - loss: 2.3208
Epoch 6/20
4/4 - 0s - 8ms/step - loss: 2.2633
Epoch 7/20
4/4 - 0s - 7ms/step - loss: 2.2212
Epoch 8/20
4/4 - 0s - 7ms/step - loss: 2.1784
Epoch 9/20
4/4 - 0s - 8ms/step - loss: 2.1265
Epoch 10/20
4/4 - 0s - 8ms/step - loss: 2.0814
Epoch 11/20
4/4 - 0s - 11ms/step - loss: 2.0429
Epoch 12/20
4/4 - 0s - 9ms/step - loss: 2.0016
Epoch 13/20
4/4 - 0s - 12ms/step - loss: 1.9663
Epoch 14/20
4/4 - 0s - 12ms/step - loss: 1.9342
Epoch 15/20
4/4 - 0s - 11ms/step - loss: 1.9075
Epoch 16/20
4/4 - 0s - 8ms/step - loss: 1.8754
Epoch 17/20
4/4 - 0s - 8ms/step - loss: 1.8475
Epoch 18/20
4/4 - 0s - 8ms/step - loss: 1.8260
Epoch 19/20
4/4 - 0s - 10ms/step - loss: 1.8038
Epoch 20/20
4/4 - 0s - 8ms/step - loss: 1.7819
2/2 - 1s - 365ms/step
[CV] END ............................l1_reg=0.001, n_units=5; total time=   4.4s
Epoch 1/20
5/5 - 3s - 696ms/step - loss: 2.2506
Epoch 2/20
5/5 - 0s - 6ms/step - loss: 2.1639
Epoch 3/20
5/5 - 0s - 7ms/step - loss: 2.0966
Epoch 4/20
5/5 - 0s - 7ms/step - loss: 2.0224
Epoch 5/20
5/5 - 0s - 6ms/step - loss: 1.9517
Epoch 6/20
5/5 - 0s - 6ms/step - loss: 1.8882
Epoch 7/20
5/5 - 0s - 6ms/step - loss: 1.8255
Epoch 8/20
5/5 - 0s - 6ms/step - loss: 1.7744
Epoch 9/20
5/5 - 0s - 7ms/step - loss: 1.7175
Epoch 10/20
5/5 - 0s - 7ms/step - loss: 1.6702
Epoch 11/20
5/5 - 0s - 9ms/step - loss: 1.6222
Epoch 12/20
5/5 - 0s - 8ms/step - loss: 1.5824
Epoch 13/20
5/5 - 0s - 7ms/step - loss: 1.5379
Epoch 14/20
5/5 - 0s - 8ms/step - loss: 1.5031
Epoch 15/20
5/5 - 0s - 6ms/step - loss: 1.4655
Epoch 16/20
5/5 - 0s - 7ms/step - loss: 1.4365
Epoch 17/20
5/5 - 0s - 8ms/step - loss: 1.4043
Epoch 18/20
5/5 - 0s - 7ms/step - loss: 1.3780
Epoch 19/20
5/5 - 0s - 9ms/step - loss: 1.3547
Epoch 20/20
5/5 - 0s - 8ms/step - loss: 1.3301
2/2 - 1s - 338ms/step
[CV] END ............................l1_reg=0.001, n_units=5; total time=   4.9s
Epoch 1/20
7/7 - 3s - 396ms/step - loss: 2.2067
Epoch 2/20
7/7 - 0s - 6ms/step - loss: 2.0948
Epoch 3/20
7/7 - 0s - 6ms/step - loss: 1.9998
Epoch 4/20
7/7 - 0s - 7ms/step - loss: 1.9180
Epoch 5/20
7/7 - 0s - 5ms/step - loss: 1.8383
Epoch 6/20
7/7 - 0s - 5ms/step - loss: 1.7522
Epoch 7/20
7/7 - 0s - 6ms/step - loss: 1.6876
Epoch 8/20
7/7 - 0s - 6ms/step - loss: 1.6163
Epoch 9/20
7/7 - 0s - 11ms/step - loss: 1.5542
Epoch 10/20
7/7 - 0s - 7ms/step - loss: 1.5070
Epoch 11/20
7/7 - 0s - 11ms/step - loss: 1.4626
Epoch 12/20
7/7 - 0s - 7ms/step - loss: 1.4162
Epoch 13/20
7/7 - 0s - 6ms/step - loss: 1.3700
Epoch 14/20
7/7 - 0s - 6ms/step - loss: 1.3337
Epoch 15/20
7/7 - 0s - 7ms/step - loss: 1.3044
Epoch 16/20
7/7 - 0s - 8ms/step - loss: 1.2771
Epoch 17/20
7/7 - 0s - 6ms/step - loss: 1.2495
Epoch 18/20
7/7 - 0s - 6ms/step - loss: 1.2234
Epoch 19/20
7/7 - 0s - 6ms/step - loss: 1.1984
Epoch 20/20
7/7 - 0s - 6ms/step - loss: 1.1785
2/2 - 1s - 477ms/step
[CV] END ............................l1_reg=0.001, n_units=5; total time=   4.6s
Epoch 1/20
8/8 - 3s - 357ms/step - loss: 2.0563
Epoch 2/20
8/8 - 0s - 4ms/step - loss: 1.9340
Epoch 3/20
8/8 - 0s - 4ms/step - loss: 1.8166
Epoch 4/20
8/8 - 0s - 5ms/step - loss: 1.7054
Epoch 5/20
8/8 - 0s - 5ms/step - loss: 1.6086
Epoch 6/20
8/8 - 0s - 5ms/step - loss: 1.5296
Epoch 7/20
8/8 - 0s - 6ms/step - loss: 1.4442
Epoch 8/20
8/8 - 0s - 5ms/step - loss: 1.3779
Epoch 9/20
8/8 - 0s - 7ms/step - loss: 1.3105
Epoch 10/20
8/8 - 0s - 5ms/step - loss: 1.2581
Epoch 11/20
8/8 - 0s - 5ms/step - loss: 1.2079
Epoch 12/20
8/8 - 0s - 6ms/step - loss: 1.1648
Epoch 13/20
8/8 - 0s - 5ms/step - loss: 1.1283
Epoch 14/20
8/8 - 0s - 7ms/step - loss: 1.0954
Epoch 15/20
8/8 - 0s - 7ms/step - loss: 1.0695
Epoch 16/20
8/8 - 0s - 7ms/step - loss: 1.0459
Epoch 17/20
8/8 - 0s - 7ms/step - loss: 1.0298
Epoch 18/20
8/8 - 0s - 7ms/step - loss: 1.0080
Epoch 19/20
8/8 - 0s - 5ms/step - loss: 0.9983
Epoch 20/20
8/8 - 0s - 8ms/step - loss: 0.9851
2/2 - 1s - 339ms/step
[CV] END ............................l1_reg=0.001, n_units=5; total time=   4.5s
Epoch 1/20
2/2 - 3s - 1s/step - loss: 2.8509
Epoch 2/20
2/2 - 0s - 22ms/step - loss: 2.7736
Epoch 3/20
2/2 - 0s - 15ms/step - loss: 2.7087
Epoch 4/20
2/2 - 0s - 12ms/step - loss: 2.6568
Epoch 5/20
2/2 - 0s - 12ms/step - loss: 2.5919
Epoch 6/20
2/2 - 0s - 14ms/step - loss: 2.5332
Epoch 7/20
2/2 - 0s - 14ms/step - loss: 2.4765
Epoch 8/20
2/2 - 0s - 12ms/step - loss: 2.4195
Epoch 9/20
2/2 - 0s - 13ms/step - loss: 2.3613
Epoch 10/20
2/2 - 0s - 14ms/step - loss: 2.3148
Epoch 11/20
2/2 - 0s - 19ms/step - loss: 2.2646
Epoch 12/20
2/2 - 0s - 15ms/step - loss: 2.2099
Epoch 13/20
2/2 - 0s - 16ms/step - loss: 2.1681
Epoch 14/20
2/2 - 0s - 13ms/step - loss: 2.1266
Epoch 15/20
2/2 - 0s - 13ms/step - loss: 2.0766
Epoch 16/20
2/2 - 0s - 20ms/step - loss: 2.0356
Epoch 17/20
2/2 - 0s - 13ms/step - loss: 1.9948
Epoch 18/20
2/2 - 0s - 17ms/step - loss: 1.9560
Epoch 19/20
2/2 - 0s - 22ms/step - loss: 1.9157
Epoch 20/20
2/2 - 0s - 17ms/step - loss: 1.8810
2/2 - 1s - 351ms/step
[CV] END ...........................l1_reg=0.001, n_units=10; total time=   4.1s
Epoch 1/20
4/4 - 3s - 691ms/step - loss: 2.9777
Epoch 2/20
4/4 - 0s - 7ms/step - loss: 2.8509
Epoch 3/20
4/4 - 0s - 9ms/step - loss: 2.7312
Epoch 4/20
4/4 - 0s - 7ms/step - loss: 2.6180
Epoch 5/20
4/4 - 0s - 8ms/step - loss: 2.5234
Epoch 6/20
4/4 - 0s - 7ms/step - loss: 2.4253
Epoch 7/20
4/4 - 0s - 9ms/step - loss: 2.3501
Epoch 8/20
4/4 - 0s - 9ms/step - loss: 2.2785
Epoch 9/20
4/4 - 0s - 10ms/step - loss: 2.2013
Epoch 10/20
4/4 - 0s - 11ms/step - loss: 2.1309
Epoch 11/20
4/4 - 0s - 9ms/step - loss: 2.0662
Epoch 12/20
4/4 - 0s - 12ms/step - loss: 1.9876
Epoch 13/20
4/4 - 0s - 8ms/step - loss: 1.9242
Epoch 14/20
4/4 - 0s - 8ms/step - loss: 1.8706
Epoch 15/20
4/4 - 0s - 9ms/step - loss: 1.8197
Epoch 16/20
4/4 - 0s - 8ms/step - loss: 1.7729
Epoch 17/20
4/4 - 0s - 12ms/step - loss: 1.7283
Epoch 18/20
4/4 - 0s - 13ms/step - loss: 1.6801
Epoch 19/20
4/4 - 0s - 9ms/step - loss: 1.6405
Epoch 20/20
4/4 - 0s - 12ms/step - loss: 1.6056
2/2 - 1s - 351ms/step
[CV] END ...........................l1_reg=0.001, n_units=10; total time=   4.2s
Epoch 1/20
5/5 - 3s - 553ms/step - loss: 2.6963
Epoch 2/20
5/5 - 0s - 6ms/step - loss: 2.5149
Epoch 3/20
5/5 - 0s - 8ms/step - loss: 2.3724
Epoch 4/20
5/5 - 0s - 7ms/step - loss: 2.2439
Epoch 5/20
5/5 - 0s - 6ms/step - loss: 2.0941
Epoch 6/20
5/5 - 0s - 8ms/step - loss: 1.9908
Epoch 7/20
5/5 - 0s - 7ms/step - loss: 1.8827
Epoch 8/20
5/5 - 0s - 7ms/step - loss: 1.7756
Epoch 9/20
5/5 - 0s - 9ms/step - loss: 1.6929
Epoch 10/20
5/5 - 0s - 10ms/step - loss: 1.6025
Epoch 11/20
5/5 - 0s - 11ms/step - loss: 1.5383
Epoch 12/20
5/5 - 0s - 9ms/step - loss: 1.4731
Epoch 13/20
5/5 - 0s - 7ms/step - loss: 1.4200
Epoch 14/20
5/5 - 0s - 8ms/step - loss: 1.3638
Epoch 15/20
5/5 - 0s - 7ms/step - loss: 1.3192
Epoch 16/20
5/5 - 0s - 8ms/step - loss: 1.2831
Epoch 17/20
5/5 - 0s - 10ms/step - loss: 1.2481
Epoch 18/20
5/5 - 0s - 9ms/step - loss: 1.2286
Epoch 19/20
5/5 - 0s - 8ms/step - loss: 1.2023
Epoch 20/20
5/5 - 0s - 8ms/step - loss: 1.1906
2/2 - 1s - 353ms/step
[CV] END ...........................l1_reg=0.001, n_units=10; total time=   4.3s
Epoch 1/20
7/7 - 3s - 394ms/step - loss: 2.6356
Epoch 2/20
7/7 - 0s - 10ms/step - loss: 2.4172
Epoch 3/20
7/7 - 0s - 5ms/step - loss: 2.1991
Epoch 4/20
7/7 - 0s - 6ms/step - loss: 2.0113
Epoch 5/20
7/7 - 0s - 5ms/step - loss: 1.8511
Epoch 6/20
7/7 - 0s - 6ms/step - loss: 1.7119
Epoch 7/20
7/7 - 0s - 5ms/step - loss: 1.5938
Epoch 8/20
7/7 - 0s - 7ms/step - loss: 1.4745
Epoch 9/20
7/7 - 0s - 7ms/step - loss: 1.3873
Epoch 10/20
7/7 - 0s - 6ms/step - loss: 1.3155
Epoch 11/20
7/7 - 0s - 6ms/step - loss: 1.2451
Epoch 12/20
7/7 - 0s - 6ms/step - loss: 1.1926
Epoch 13/20
7/7 - 0s - 5ms/step - loss: 1.1473
Epoch 14/20
7/7 - 0s - 6ms/step - loss: 1.1153
Epoch 15/20
7/7 - 0s - 7ms/step - loss: 1.0953
Epoch 16/20
7/7 - 0s - 6ms/step - loss: 1.0813
Epoch 17/20
7/7 - 0s - 6ms/step - loss: 1.0672
Epoch 18/20
7/7 - 0s - 8ms/step - loss: 1.0703
Epoch 19/20
7/7 - 0s - 9ms/step - loss: 1.0630
Epoch 20/20
7/7 - 0s - 6ms/step - loss: 1.0618
2/2 - 1s - 342ms/step
[CV] END ...........................l1_reg=0.001, n_units=10; total time=   4.3s
Epoch 1/20
8/8 - 3s - 355ms/step - loss: 2.5116
Epoch 2/20
8/8 - 0s - 5ms/step - loss: 2.2448
Epoch 3/20
8/8 - 0s - 5ms/step - loss: 2.0240
Epoch 4/20
8/8 - 0s - 5ms/step - loss: 1.8216
Epoch 5/20
8/8 - 0s - 6ms/step - loss: 1.6435
Epoch 6/20
8/8 - 0s - 7ms/step - loss: 1.4920
Epoch 7/20
8/8 - 0s - 6ms/step - loss: 1.3740
Epoch 8/20
8/8 - 0s - 6ms/step - loss: 1.2598
Epoch 9/20
8/8 - 0s - 6ms/step - loss: 1.1713
Epoch 10/20
8/8 - 0s - 6ms/step - loss: 1.1074
Epoch 11/20
8/8 - 0s - 6ms/step - loss: 1.0550
Epoch 12/20
8/8 - 0s - 6ms/step - loss: 1.0095
Epoch 13/20
8/8 - 0s - 5ms/step - loss: 0.9848
Epoch 14/20
8/8 - 0s - 6ms/step - loss: 0.9670
Epoch 15/20
8/8 - 0s - 6ms/step - loss: 0.9559
Epoch 16/20
8/8 - 0s - 5ms/step - loss: 0.9524
Epoch 17/20
8/8 - 0s - 7ms/step - loss: 0.9477
Epoch 18/20
8/8 - 0s - 9ms/step - loss: 0.9476
Epoch 19/20
8/8 - 0s - 8ms/step - loss: 0.9436
Epoch 20/20
8/8 - 0s - 9ms/step - loss: 0.9435
2/2 - 1s - 444ms/step
[CV] END ...........................l1_reg=0.001, n_units=10; total time=   4.7s
Epoch 1/20
2/2 - 3s - 2s/step - loss: 2.3292
Epoch 2/20
2/2 - 0s - 12ms/step - loss: 2.2538
Epoch 3/20
2/2 - 0s - 17ms/step - loss: 2.1876
Epoch 4/20
2/2 - 0s - 14ms/step - loss: 2.1219
Epoch 5/20
2/2 - 0s - 12ms/step - loss: 2.0681
Epoch 6/20
2/2 - 0s - 13ms/step - loss: 2.0237
Epoch 7/20
2/2 - 0s - 13ms/step - loss: 1.9648
Epoch 8/20
2/2 - 0s - 14ms/step - loss: 1.9197
Epoch 9/20
2/2 - 0s - 15ms/step - loss: 1.8628
Epoch 10/20
2/2 - 0s - 18ms/step - loss: 1.8248
Epoch 11/20
2/2 - 0s - 20ms/step - loss: 1.8083
Epoch 12/20
2/2 - 0s - 15ms/step - loss: 1.7586
Epoch 13/20
2/2 - 0s - 17ms/step - loss: 1.7252
Epoch 14/20
2/2 - 0s - 18ms/step - loss: 1.6899
Epoch 15/20
2/2 - 0s - 21ms/step - loss: 1.6698
Epoch 16/20
2/2 - 0s - 20ms/step - loss: 1.6423
Epoch 17/20
2/2 - 0s - 17ms/step - loss: 1.6196
Epoch 18/20
2/2 - 0s - 21ms/step - loss: 1.5871
Epoch 19/20
2/2 - 0s - 23ms/step - loss: 1.5733
Epoch 20/20
2/2 - 0s - 18ms/step - loss: 1.5445
2/2 - 1s - 345ms/step
[CV] END ...........................l1_reg=0.001, n_units=20; total time=   4.8s
Epoch 1/20
4/4 - 3s - 717ms/step - loss: 2.3003
Epoch 2/20
4/4 - 0s - 7ms/step - loss: 2.1676
Epoch 3/20
4/4 - 0s - 7ms/step - loss: 2.0539
Epoch 4/20
4/4 - 0s - 9ms/step - loss: 1.9562
Epoch 5/20
4/4 - 0s - 8ms/step - loss: 1.8845
Epoch 6/20
4/4 - 0s - 9ms/step - loss: 1.8134
Epoch 7/20
4/4 - 0s - 9ms/step - loss: 1.7504
Epoch 8/20
4/4 - 0s - 8ms/step - loss: 1.6971
Epoch 9/20
4/4 - 0s - 10ms/step - loss: 1.6347
Epoch 10/20
4/4 - 0s - 12ms/step - loss: 1.6014
Epoch 11/20
4/4 - 0s - 10ms/step - loss: 1.5553
Epoch 12/20
4/4 - 0s - 11ms/step - loss: 1.5317
Epoch 13/20
4/4 - 0s - 8ms/step - loss: 1.4963
Epoch 14/20
4/4 - 0s - 9ms/step - loss: 1.4786
Epoch 15/20
4/4 - 0s - 11ms/step - loss: 1.4533
Epoch 16/20
4/4 - 0s - 9ms/step - loss: 1.4486
Epoch 17/20
4/4 - 0s - 10ms/step - loss: 1.4273
Epoch 18/20
4/4 - 0s - 9ms/step - loss: 1.4260
Epoch 19/20
4/4 - 0s - 11ms/step - loss: 1.4231
Epoch 20/20
4/4 - 0s - 12ms/step - loss: 1.4192
2/2 - 1s - 348ms/step
[CV] END ...........................l1_reg=0.001, n_units=20; total time=   4.3s
Epoch 1/20
5/5 - 3s - 543ms/step - loss: 1.9864
Epoch 2/20
5/5 - 0s - 6ms/step - loss: 1.8108
Epoch 3/20
5/5 - 0s - 8ms/step - loss: 1.6805
Epoch 4/20
5/5 - 0s - 7ms/step - loss: 1.5514
Epoch 5/20
5/5 - 0s - 7ms/step - loss: 1.4586
Epoch 6/20
5/5 - 0s - 7ms/step - loss: 1.3611
Epoch 7/20
5/5 - 0s - 7ms/step - loss: 1.3062
Epoch 8/20
5/5 - 0s - 8ms/step - loss: 1.2405
Epoch 9/20
5/5 - 0s - 7ms/step - loss: 1.2183
Epoch 10/20
5/5 - 0s - 7ms/step - loss: 1.1836
Epoch 11/20
5/5 - 0s - 8ms/step - loss: 1.1753
Epoch 12/20
5/5 - 0s - 7ms/step - loss: 1.1617
Epoch 13/20
5/5 - 0s - 8ms/step - loss: 1.1631
Epoch 14/20
5/5 - 0s - 8ms/step - loss: 1.1588
Epoch 15/20
5/5 - 0s - 8ms/step - loss: 1.1624
Epoch 16/20
5/5 - 0s - 6ms/step - loss: 1.1563
Epoch 17/20
5/5 - 0s - 9ms/step - loss: 1.1524
Epoch 18/20
5/5 - 0s - 9ms/step - loss: 1.1522
Epoch 19/20
5/5 - 0s - 10ms/step - loss: 1.1485
Epoch 20/20
5/5 - 0s - 11ms/step - loss: 1.1460
2/2 - 1s - 345ms/step
[CV] END ...........................l1_reg=0.001, n_units=20; total time=   4.2s
Epoch 1/20
7/7 - 3s - 387ms/step - loss: 1.8876
Epoch 2/20
7/7 - 0s - 5ms/step - loss: 1.6940
Epoch 3/20
7/7 - 0s - 5ms/step - loss: 1.5378
Epoch 4/20
7/7 - 0s - 6ms/step - loss: 1.4212
Epoch 5/20
7/7 - 0s - 7ms/step - loss: 1.3197
Epoch 6/20
7/7 - 0s - 5ms/step - loss: 1.2469
Epoch 7/20
7/7 - 0s - 5ms/step - loss: 1.1933
Epoch 8/20
7/7 - 0s - 5ms/step - loss: 1.1453
Epoch 9/20
7/7 - 0s - 5ms/step - loss: 1.1194
Epoch 10/20
7/7 - 0s - 6ms/step - loss: 1.1001
Epoch 11/20
7/7 - 0s - 6ms/step - loss: 1.0862
Epoch 12/20
7/7 - 0s - 6ms/step - loss: 1.0853
Epoch 13/20
7/7 - 0s - 6ms/step - loss: 1.0795
Epoch 14/20
7/7 - 0s - 8ms/step - loss: 1.0761
Epoch 15/20
7/7 - 0s - 7ms/step - loss: 1.0651
Epoch 16/20
7/7 - 0s - 7ms/step - loss: 1.0654
Epoch 17/20
7/7 - 0s - 7ms/step - loss: 1.0665
Epoch 18/20
7/7 - 0s - 6ms/step - loss: 1.0654
Epoch 19/20
7/7 - 0s - 7ms/step - loss: 1.0592
Epoch 20/20
7/7 - 0s - 6ms/step - loss: 1.0614
2/2 - 1s - 333ms/step
[CV] END ...........................l1_reg=0.001, n_units=20; total time=   4.2s
Epoch 1/20
8/8 - 3s - 345ms/step - loss: 1.7190
Epoch 2/20
8/8 - 0s - 5ms/step - loss: 1.4612
Epoch 3/20
8/8 - 0s - 5ms/step - loss: 1.2725
Epoch 4/20
8/8 - 0s - 5ms/step - loss: 1.1389
Epoch 5/20
8/8 - 0s - 6ms/step - loss: 1.0201
Epoch 6/20
8/8 - 0s - 5ms/step - loss: 0.9764
Epoch 7/20
8/8 - 0s - 7ms/step - loss: 0.9587
Epoch 8/20
8/8 - 0s - 7ms/step - loss: 0.9579
Epoch 9/20
8/8 - 0s - 6ms/step - loss: 0.9582
Epoch 10/20
8/8 - 0s - 5ms/step - loss: 0.9530
Epoch 11/20
8/8 - 0s - 6ms/step - loss: 0.9435
Epoch 12/20
8/8 - 0s - 5ms/step - loss: 0.9440
Epoch 13/20
8/8 - 0s - 6ms/step - loss: 0.9407
Epoch 14/20
8/8 - 0s - 6ms/step - loss: 0.9399
Epoch 15/20
8/8 - 0s - 8ms/step - loss: 0.9369
Epoch 16/20
8/8 - 0s - 7ms/step - loss: 0.9352
Epoch 17/20
8/8 - 0s - 7ms/step - loss: 0.9408
Epoch 18/20
8/8 - 0s - 8ms/step - loss: 0.9349
Epoch 19/20
8/8 - 0s - 7ms/step - loss: 0.9331
Epoch 20/20
8/8 - 0s - 6ms/step - loss: 0.9339
2/2 - 1s - 399ms/step
[CV] END ...........................l1_reg=0.001, n_units=20; total time=   4.5s
Epoch 1/20
2/2 - 3s - 1s/step - loss: 2.5928
Epoch 2/20
2/2 - 0s - 14ms/step - loss: 2.5589
Epoch 3/20
2/2 - 0s - 13ms/step - loss: 2.5255
Epoch 4/20
2/2 - 0s - 13ms/step - loss: 2.4976
Epoch 5/20
2/2 - 0s - 12ms/step - loss: 2.4682
Epoch 6/20
2/2 - 0s - 13ms/step - loss: 2.4317
Epoch 7/20
2/2 - 0s - 14ms/step - loss: 2.4069
Epoch 8/20
2/2 - 0s - 14ms/step - loss: 2.3763
Epoch 9/20
2/2 - 0s - 20ms/step - loss: 2.3472
Epoch 10/20
2/2 - 0s - 15ms/step - loss: 2.3232
Epoch 11/20
2/2 - 0s - 22ms/step - loss: 2.2963
Epoch 12/20
2/2 - 0s - 12ms/step - loss: 2.2707
Epoch 13/20
2/2 - 0s - 13ms/step - loss: 2.2435
Epoch 14/20
2/2 - 0s - 15ms/step - loss: 2.2220
Epoch 15/20
2/2 - 0s - 17ms/step - loss: 2.1970
Epoch 16/20
2/2 - 0s - 21ms/step - loss: 2.1694
Epoch 17/20
2/2 - 0s - 15ms/step - loss: 2.1474
Epoch 18/20
2/2 - 0s - 18ms/step - loss: 2.1268
Epoch 19/20
2/2 - 0s - 20ms/step - loss: 2.1046
Epoch 20/20
2/2 - 0s - 81ms/step - loss: 2.0831
2/2 - 1s - 372ms/step
[CV] END .............................l1_reg=0.01, n_units=5; total time=   4.5s
Epoch 1/20
4/4 - 3s - 689ms/step - loss: 2.6234
Epoch 2/20
4/4 - 0s - 7ms/step - loss: 2.5592
Epoch 3/20
4/4 - 0s - 9ms/step - loss: 2.5106
Epoch 4/20
4/4 - 0s - 8ms/step - loss: 2.4618
Epoch 5/20
4/4 - 0s - 8ms/step - loss: 2.4125
Epoch 6/20
4/4 - 0s - 7ms/step - loss: 2.3607
Epoch 7/20
4/4 - 0s - 8ms/step - loss: 2.3137
Epoch 8/20
4/4 - 0s - 7ms/step - loss: 2.2640
Epoch 9/20
4/4 - 0s - 8ms/step - loss: 2.2274
Epoch 10/20
4/4 - 0s - 10ms/step - loss: 2.1831
Epoch 11/20
4/4 - 0s - 7ms/step - loss: 2.1500
Epoch 12/20
4/4 - 0s - 9ms/step - loss: 2.1112
Epoch 13/20
4/4 - 0s - 9ms/step - loss: 2.0795
Epoch 14/20
4/4 - 0s - 12ms/step - loss: 2.0486
Epoch 15/20
4/4 - 0s - 13ms/step - loss: 2.0164
Epoch 16/20
4/4 - 0s - 12ms/step - loss: 1.9921
Epoch 17/20
4/4 - 0s - 9ms/step - loss: 1.9682
Epoch 18/20
4/4 - 0s - 7ms/step - loss: 1.9348
Epoch 19/20
4/4 - 0s - 7ms/step - loss: 1.9071
Epoch 20/20
4/4 - 0s - 8ms/step - loss: 1.8772
2/2 - 1s - 344ms/step
[CV] END .............................l1_reg=0.01, n_units=5; total time=   4.1s
Epoch 1/20
5/5 - 3s - 602ms/step - loss: 2.3130
Epoch 2/20
5/5 - 0s - 6ms/step - loss: 2.2339
Epoch 3/20
5/5 - 0s - 8ms/step - loss: 2.1588
Epoch 4/20
5/5 - 0s - 6ms/step - loss: 2.0839
Epoch 5/20
5/5 - 0s - 7ms/step - loss: 2.0152
Epoch 6/20
5/5 - 0s - 7ms/step - loss: 1.9533
Epoch 7/20
5/5 - 0s - 8ms/step - loss: 1.8937
Epoch 8/20
5/5 - 0s - 9ms/step - loss: 1.8421
Epoch 9/20
5/5 - 0s - 9ms/step - loss: 1.7870
Epoch 10/20
5/5 - 0s - 7ms/step - loss: 1.7352
Epoch 11/20
5/5 - 0s - 8ms/step - loss: 1.6919
Epoch 12/20
5/5 - 0s - 8ms/step - loss: 1.6476
Epoch 13/20
5/5 - 0s - 8ms/step - loss: 1.6047
Epoch 14/20
5/5 - 0s - 7ms/step - loss: 1.5723
Epoch 15/20
5/5 - 0s - 8ms/step - loss: 1.5344
Epoch 16/20
5/5 - 0s - 10ms/step - loss: 1.5052
Epoch 17/20
5/5 - 0s - 9ms/step - loss: 1.4742
Epoch 18/20
5/5 - 0s - 8ms/step - loss: 1.4470
Epoch 19/20
5/5 - 0s - 7ms/step - loss: 1.4186
Epoch 20/20
5/5 - 0s - 9ms/step - loss: 1.3955
2/2 - 1s - 400ms/step
[CV] END .............................l1_reg=0.01, n_units=5; total time=   4.6s
Epoch 1/20
7/7 - 3s - 390ms/step - loss: 2.2687
Epoch 2/20
7/7 - 0s - 5ms/step - loss: 2.1590
Epoch 3/20
7/7 - 0s - 6ms/step - loss: 2.0612
Epoch 4/20
7/7 - 0s - 6ms/step - loss: 1.9709
Epoch 5/20
7/7 - 0s - 6ms/step - loss: 1.8981
Epoch 6/20
7/7 - 0s - 6ms/step - loss: 1.8102
Epoch 7/20
7/7 - 0s - 7ms/step - loss: 1.7420
Epoch 8/20
7/7 - 0s - 7ms/step - loss: 1.6671
Epoch 9/20
7/7 - 0s - 6ms/step - loss: 1.6022
Epoch 10/20
7/7 - 0s - 8ms/step - loss: 1.5492
Epoch 11/20
7/7 - 0s - 6ms/step - loss: 1.4975
Epoch 12/20
7/7 - 0s - 8ms/step - loss: 1.4606
Epoch 13/20
7/7 - 0s - 7ms/step - loss: 1.4243
Epoch 14/20
7/7 - 0s - 7ms/step - loss: 1.3887
Epoch 15/20
7/7 - 0s - 6ms/step - loss: 1.3560
Epoch 16/20
7/7 - 0s - 7ms/step - loss: 1.3235
Epoch 17/20
7/7 - 0s - 9ms/step - loss: 1.3017
Epoch 18/20
7/7 - 0s - 8ms/step - loss: 1.2802
Epoch 19/20
7/7 - 0s - 8ms/step - loss: 1.2609
Epoch 20/20
7/7 - 0s - 8ms/step - loss: 1.2486
2/2 - 1s - 347ms/step
[CV] END .............................l1_reg=0.01, n_units=5; total time=   4.4s
Epoch 1/20
8/8 - 4s - 438ms/step - loss: 2.1283
Epoch 2/20
8/8 - 0s - 5ms/step - loss: 2.0115
Epoch 3/20
8/8 - 0s - 5ms/step - loss: 1.8907
Epoch 4/20
8/8 - 0s - 6ms/step - loss: 1.7941
Epoch 5/20
8/8 - 0s - 5ms/step - loss: 1.6936
Epoch 6/20
8/8 - 0s - 7ms/step - loss: 1.6070
Epoch 7/20
8/8 - 0s - 5ms/step - loss: 1.5242
Epoch 8/20
8/8 - 0s - 6ms/step - loss: 1.4598
Epoch 9/20
8/8 - 0s - 6ms/step - loss: 1.3883
Epoch 10/20
8/8 - 0s - 5ms/step - loss: 1.3341
Epoch 11/20
8/8 - 0s - 5ms/step - loss: 1.2859
Epoch 12/20
8/8 - 0s - 5ms/step - loss: 1.2367
Epoch 13/20
8/8 - 0s - 6ms/step - loss: 1.2022
Epoch 14/20
8/8 - 0s - 6ms/step - loss: 1.1727
Epoch 15/20
8/8 - 0s - 6ms/step - loss: 1.1402
Epoch 16/20
8/8 - 0s - 5ms/step - loss: 1.1149
Epoch 17/20
8/8 - 0s - 6ms/step - loss: 1.0962
Epoch 18/20
8/8 - 0s - 5ms/step - loss: 1.0826
Epoch 19/20
8/8 - 0s - 6ms/step - loss: 1.0675
Epoch 20/20
8/8 - 0s - 6ms/step - loss: 1.0570
2/2 - 1s - 345ms/step
[CV] END .............................l1_reg=0.01, n_units=5; total time=   5.1s
Epoch 1/20
2/2 - 3s - 1s/step - loss: 2.9456
Epoch 2/20
2/2 - 0s - 16ms/step - loss: 2.8737
Epoch 3/20
2/2 - 0s - 18ms/step - loss: 2.8028
Epoch 4/20
2/2 - 0s - 13ms/step - loss: 2.7428
Epoch 5/20
2/2 - 0s - 13ms/step - loss: 2.6764
Epoch 6/20
2/2 - 0s - 13ms/step - loss: 2.6136
Epoch 7/20
2/2 - 0s - 13ms/step - loss: 2.5691
Epoch 8/20
2/2 - 0s - 14ms/step - loss: 2.5050
Epoch 9/20
2/2 - 0s - 19ms/step - loss: 2.4523
Epoch 10/20
2/2 - 0s - 13ms/step - loss: 2.4043
Epoch 11/20
2/2 - 0s - 55ms/step - loss: 2.3488
Epoch 12/20
2/2 - 0s - 15ms/step - loss: 2.3042
Epoch 13/20
2/2 - 0s - 13ms/step - loss: 2.2643
Epoch 14/20
2/2 - 0s - 16ms/step - loss: 2.2166
Epoch 15/20
2/2 - 0s - 14ms/step - loss: 2.1726
Epoch 16/20
2/2 - 0s - 15ms/step - loss: 2.1348
Epoch 17/20
2/2 - 0s - 17ms/step - loss: 2.0944
Epoch 18/20
2/2 - 0s - 22ms/step - loss: 2.0523
Epoch 19/20
2/2 - 0s - 20ms/step - loss: 2.0173
Epoch 20/20
2/2 - 0s - 23ms/step - loss: 1.9867
2/2 - 1s - 366ms/step
[CV] END ............................l1_reg=0.01, n_units=10; total time=   4.2s
Epoch 1/20
4/4 - 3s - 690ms/step - loss: 3.0758
Epoch 2/20
4/4 - 0s - 9ms/step - loss: 2.9524
Epoch 3/20
4/4 - 0s - 8ms/step - loss: 2.8210
Epoch 4/20
4/4 - 0s - 9ms/step - loss: 2.7110
Epoch 5/20
4/4 - 0s - 8ms/step - loss: 2.5883
Epoch 6/20
4/4 - 0s - 8ms/step - loss: 2.5038
Epoch 7/20
4/4 - 0s - 9ms/step - loss: 2.3980
Epoch 8/20
4/4 - 0s - 8ms/step - loss: 2.3088
Epoch 9/20
4/4 - 0s - 9ms/step - loss: 2.2244
Epoch 10/20
4/4 - 0s - 9ms/step - loss: 2.1457
Epoch 11/20
4/4 - 0s - 8ms/step - loss: 2.0783
Epoch 12/20
4/4 - 0s - 9ms/step - loss: 2.0155
Epoch 13/20
4/4 - 0s - 13ms/step - loss: 1.9543
Epoch 14/20
4/4 - 0s - 11ms/step - loss: 1.9054
Epoch 15/20
4/4 - 0s - 11ms/step - loss: 1.8597
Epoch 16/20
4/4 - 0s - 12ms/step - loss: 1.8167
Epoch 17/20
4/4 - 0s - 10ms/step - loss: 1.7715
Epoch 18/20
4/4 - 0s - 9ms/step - loss: 1.7484
Epoch 19/20
4/4 - 0s - 9ms/step - loss: 1.7024
Epoch 20/20
4/4 - 0s - 10ms/step - loss: 1.6809
2/2 - 1s - 367ms/step
[CV] END ............................l1_reg=0.01, n_units=10; total time=   4.3s
Epoch 1/20
5/5 - 3s - 560ms/step - loss: 2.7836
Epoch 2/20
5/5 - 0s - 7ms/step - loss: 2.6126
Epoch 3/20
5/5 - 0s - 8ms/step - loss: 2.4647
Epoch 4/20
5/5 - 0s - 6ms/step - loss: 2.3289
Epoch 5/20
5/5 - 0s - 7ms/step - loss: 2.1941
Epoch 6/20
5/5 - 0s - 7ms/step - loss: 2.0779
Epoch 7/20
5/5 - 0s - 8ms/step - loss: 1.9651
Epoch 8/20
5/5 - 0s - 7ms/step - loss: 1.8634
Epoch 9/20
5/5 - 0s - 10ms/step - loss: 1.7824
Epoch 10/20
5/5 - 0s - 11ms/step - loss: 1.6939
Epoch 11/20
5/5 - 0s - 10ms/step - loss: 1.6140
Epoch 12/20
5/5 - 0s - 11ms/step - loss: 1.5531
Epoch 13/20
5/5 - 0s - 6ms/step - loss: 1.4981
Epoch 14/20
5/5 - 0s - 9ms/step - loss: 1.4456
Epoch 15/20
5/5 - 0s - 8ms/step - loss: 1.4066
Epoch 16/20
5/5 - 0s - 6ms/step - loss: 1.3642
Epoch 17/20
5/5 - 0s - 7ms/step - loss: 1.3327
Epoch 18/20
5/5 - 0s - 9ms/step - loss: 1.3120
Epoch 19/20
5/5 - 0s - 11ms/step - loss: 1.2899
Epoch 20/20
5/5 - 0s - 8ms/step - loss: 1.2722
2/2 - 1s - 348ms/step
[CV] END ............................l1_reg=0.01, n_units=10; total time=   4.4s
Epoch 1/20
7/7 - 3s - 392ms/step - loss: 2.7340
Epoch 2/20
7/7 - 0s - 7ms/step - loss: 2.5135
Epoch 3/20
7/7 - 0s - 6ms/step - loss: 2.3226
Epoch 4/20
7/7 - 0s - 6ms/step - loss: 2.1330
Epoch 5/20
7/7 - 0s - 5ms/step - loss: 1.9860
Epoch 6/20
7/7 - 0s - 6ms/step - loss: 1.8341
Epoch 7/20
7/7 - 0s - 6ms/step - loss: 1.7068
Epoch 8/20
7/7 - 0s - 6ms/step - loss: 1.6110
Epoch 9/20
7/7 - 0s - 6ms/step - loss: 1.5161
Epoch 10/20
7/7 - 0s - 7ms/step - loss: 1.4571
Epoch 11/20
7/7 - 0s - 7ms/step - loss: 1.4016
Epoch 12/20
7/7 - 0s - 7ms/step - loss: 1.3583
Epoch 13/20
7/7 - 0s - 7ms/step - loss: 1.3100
Epoch 14/20
7/7 - 0s - 7ms/step - loss: 1.2619
Epoch 15/20
7/7 - 0s - 6ms/step - loss: 1.2292
Epoch 16/20
7/7 - 0s - 8ms/step - loss: 1.2080
Epoch 17/20
7/7 - 0s - 7ms/step - loss: 1.1738
Epoch 18/20
7/7 - 0s - 6ms/step - loss: 1.1687
Epoch 19/20
7/7 - 0s - 7ms/step - loss: 1.1585
Epoch 20/20
7/7 - 0s - 7ms/step - loss: 1.1574
2/2 - 1s - 342ms/step
[CV] END ............................l1_reg=0.01, n_units=10; total time=   4.3s
Epoch 1/20
8/8 - 3s - 342ms/step - loss: 2.5878
Epoch 2/20
8/8 - 0s - 5ms/step - loss: 2.3061
Epoch 3/20
8/8 - 0s - 5ms/step - loss: 2.0825
Epoch 4/20
8/8 - 0s - 5ms/step - loss: 1.8694
Epoch 5/20
8/8 - 0s - 5ms/step - loss: 1.6921
Epoch 6/20
8/8 - 0s - 6ms/step - loss: 1.5408
Epoch 7/20
8/8 - 0s - 7ms/step - loss: 1.4307
Epoch 8/20
8/8 - 0s - 7ms/step - loss: 1.3125
Epoch 9/20
8/8 - 0s - 7ms/step - loss: 1.2436
Epoch 10/20
8/8 - 0s - 7ms/step - loss: 1.1788
Epoch 11/20
8/8 - 0s - 6ms/step - loss: 1.1297
Epoch 12/20
8/8 - 0s - 5ms/step - loss: 1.0914
Epoch 13/20
8/8 - 0s - 7ms/step - loss: 1.0737
Epoch 14/20
8/8 - 0s - 7ms/step - loss: 1.0555
Epoch 15/20
8/8 - 0s - 6ms/step - loss: 1.0501
Epoch 16/20
8/8 - 0s - 5ms/step - loss: 1.0438
Epoch 17/20
8/8 - 0s - 6ms/step - loss: 1.0399
Epoch 18/20
8/8 - 0s - 6ms/step - loss: 1.0373
Epoch 19/20
8/8 - 0s - 6ms/step - loss: 1.0356
Epoch 20/20
8/8 - 0s - 7ms/step - loss: 1.0328
2/2 - 1s - 380ms/step
[CV] END ............................l1_reg=0.01, n_units=10; total time=   4.5s
Epoch 1/20
2/2 - 3s - 1s/step - loss: 2.4601
Epoch 2/20
2/2 - 0s - 14ms/step - loss: 2.3878
Epoch 3/20
2/2 - 0s - 14ms/step - loss: 2.3216
Epoch 4/20
2/2 - 0s - 13ms/step - loss: 2.2673
Epoch 5/20
2/2 - 0s - 12ms/step - loss: 2.2015
Epoch 6/20
2/2 - 0s - 16ms/step - loss: 2.1557
Epoch 7/20
2/2 - 0s - 12ms/step - loss: 2.1132
Epoch 8/20
2/2 - 0s - 14ms/step - loss: 2.0640
Epoch 9/20
2/2 - 0s - 17ms/step - loss: 2.0153
Epoch 10/20
2/2 - 0s - 21ms/step - loss: 1.9733
Epoch 11/20
2/2 - 0s - 21ms/step - loss: 1.9353
Epoch 12/20
2/2 - 0s - 19ms/step - loss: 1.9036
Epoch 13/20
2/2 - 0s - 17ms/step - loss: 1.8631
Epoch 14/20
2/2 - 0s - 22ms/step - loss: 1.8406
Epoch 15/20
2/2 - 0s - 16ms/step - loss: 1.8057
Epoch 16/20
2/2 - 0s - 15ms/step - loss: 1.7685
Epoch 17/20
2/2 - 0s - 13ms/step - loss: 1.7426
Epoch 18/20
2/2 - 0s - 15ms/step - loss: 1.7203
Epoch 19/20
2/2 - 0s - 20ms/step - loss: 1.6970
Epoch 20/20
2/2 - 0s - 19ms/step - loss: 1.6810
2/2 - 1s - 349ms/step
[CV] END ............................l1_reg=0.01, n_units=20; total time=   4.1s
Epoch 1/20
4/4 - 3s - 669ms/step - loss: 2.4405
Epoch 2/20
4/4 - 0s - 8ms/step - loss: 2.2955
Epoch 3/20
4/4 - 0s - 8ms/step - loss: 2.2027
Epoch 4/20
4/4 - 0s - 8ms/step - loss: 2.0988
Epoch 5/20
4/4 - 0s - 7ms/step - loss: 2.0322
Epoch 6/20
4/4 - 0s - 7ms/step - loss: 1.9693
Epoch 7/20
4/4 - 0s - 9ms/step - loss: 1.9112
Epoch 8/20
4/4 - 0s - 10ms/step - loss: 1.8615
Epoch 9/20
4/4 - 0s - 8ms/step - loss: 1.8006
Epoch 10/20
4/4 - 0s - 9ms/step - loss: 1.7433
Epoch 11/20
4/4 - 0s - 11ms/step - loss: 1.6915
Epoch 12/20
4/4 - 0s - 12ms/step - loss: 1.6713
Epoch 13/20
4/4 - 0s - 9ms/step - loss: 1.6328
Epoch 14/20
4/4 - 0s - 7ms/step - loss: 1.6140
Epoch 15/20
4/4 - 0s - 8ms/step - loss: 1.5903
Epoch 16/20
4/4 - 0s - 8ms/step - loss: 1.5805
Epoch 17/20
4/4 - 0s - 9ms/step - loss: 1.5700
Epoch 18/20
4/4 - 0s - 9ms/step - loss: 1.5684
Epoch 19/20
4/4 - 0s - 9ms/step - loss: 1.5592
Epoch 20/20
4/4 - 0s - 13ms/step - loss: 1.5523
2/2 - 1s - 350ms/step
[CV] END ............................l1_reg=0.01, n_units=20; total time=   4.1s
Epoch 1/20
5/5 - 3s - 549ms/step - loss: 2.1130
Epoch 2/20
5/5 - 0s - 7ms/step - loss: 1.9493
Epoch 3/20
5/5 - 0s - 7ms/step - loss: 1.8119
Epoch 4/20
5/5 - 0s - 6ms/step - loss: 1.6856
Epoch 5/20
5/5 - 0s - 6ms/step - loss: 1.5850
Epoch 6/20
5/5 - 0s - 7ms/step - loss: 1.4954
Epoch 7/20
5/5 - 0s - 7ms/step - loss: 1.4214
Epoch 8/20
5/5 - 0s - 7ms/step - loss: 1.3695
Epoch 9/20
5/5 - 0s - 7ms/step - loss: 1.3198
Epoch 10/20
5/5 - 0s - 8ms/step - loss: 1.3060
Epoch 11/20
5/5 - 0s - 7ms/step - loss: 1.2939
Epoch 12/20
5/5 - 0s - 7ms/step - loss: 1.3000
Epoch 13/20
5/5 - 0s - 8ms/step - loss: 1.2894
Epoch 14/20
5/5 - 0s - 8ms/step - loss: 1.2856
Epoch 15/20
5/5 - 0s - 8ms/step - loss: 1.2792
Epoch 16/20
5/5 - 0s - 9ms/step - loss: 1.2757
Epoch 17/20
5/5 - 0s - 10ms/step - loss: 1.2709
Epoch 18/20
5/5 - 0s - 9ms/step - loss: 1.2687
Epoch 19/20
5/5 - 0s - 9ms/step - loss: 1.2654
Epoch 20/20
5/5 - 0s - 8ms/step - loss: 1.2618
2/2 - 1s - 355ms/step
[CV] END ............................l1_reg=0.01, n_units=20; total time=   4.3s
Epoch 1/20
7/7 - 3s - 400ms/step - loss: 2.0187
Epoch 2/20
7/7 - 0s - 5ms/step - loss: 1.8033
Epoch 3/20
7/7 - 0s - 6ms/step - loss: 1.6152
Epoch 4/20
7/7 - 0s - 6ms/step - loss: 1.5032
Epoch 5/20
7/7 - 0s - 5ms/step - loss: 1.3967
Epoch 6/20
7/7 - 0s - 6ms/step - loss: 1.3242
Epoch 7/20
7/7 - 0s - 6ms/step - loss: 1.2632
Epoch 8/20
7/7 - 0s - 6ms/step - loss: 1.2255
Epoch 9/20
7/7 - 0s - 7ms/step - loss: 1.2147
Epoch 10/20
7/7 - 0s - 6ms/step - loss: 1.2111
Epoch 11/20
7/7 - 0s - 6ms/step - loss: 1.2095
Epoch 12/20
7/7 - 0s - 6ms/step - loss: 1.1990
Epoch 13/20
7/7 - 0s - 7ms/step - loss: 1.1925
Epoch 14/20
7/7 - 0s - 6ms/step - loss: 1.1823
Epoch 15/20
7/7 - 0s - 7ms/step - loss: 1.1815
Epoch 16/20
7/7 - 0s - 6ms/step - loss: 1.1765
Epoch 17/20
7/7 - 0s - 8ms/step - loss: 1.1739
Epoch 18/20
7/7 - 0s - 6ms/step - loss: 1.1703
Epoch 19/20
7/7 - 0s - 6ms/step - loss: 1.1693
Epoch 20/20
7/7 - 0s - 6ms/step - loss: 1.1642
2/2 - 1s - 347ms/step
[CV] END ............................l1_reg=0.01, n_units=20; total time=   4.4s
Epoch 1/20
8/8 - 3s - 360ms/step - loss: 1.8604
Epoch 2/20
8/8 - 0s - 7ms/step - loss: 1.6137
Epoch 3/20
8/8 - 0s - 6ms/step - loss: 1.4225
Epoch 4/20
8/8 - 0s - 8ms/step - loss: 1.2822
Epoch 5/20
8/8 - 0s - 7ms/step - loss: 1.1875
Epoch 6/20
8/8 - 0s - 6ms/step - loss: 1.1231
Epoch 7/20
8/8 - 0s - 7ms/step - loss: 1.0974
Epoch 8/20
8/8 - 0s - 7ms/step - loss: 1.0812
Epoch 9/20
8/8 - 0s - 8ms/step - loss: 1.0835
Epoch 10/20
8/8 - 0s - 9ms/step - loss: 1.0730
Epoch 11/20
8/8 - 0s - 8ms/step - loss: 1.0646
Epoch 12/20
8/8 - 0s - 7ms/step - loss: 1.0612
Epoch 13/20
8/8 - 0s - 7ms/step - loss: 1.0584
Epoch 14/20
8/8 - 0s - 7ms/step - loss: 1.0532
Epoch 15/20
8/8 - 0s - 7ms/step - loss: 1.0508
Epoch 16/20
8/8 - 0s - 6ms/step - loss: 1.0456
Epoch 17/20
8/8 - 0s - 7ms/step - loss: 1.0465
Epoch 18/20
8/8 - 0s - 7ms/step - loss: 1.0448
Epoch 19/20
8/8 - 0s - 6ms/step - loss: 1.0383
Epoch 20/20
8/8 - 0s - 7ms/step - loss: 1.0370
2/2 - 1s - 524ms/step
[CV] END ............................l1_reg=0.01, n_units=20; total time=   5.1s
Epoch 1/20
2/2 - 4s - 2s/step - loss: 3.2435
Epoch 2/20
2/2 - 0s - 11ms/step - loss: 3.2104
Epoch 3/20
2/2 - 0s - 14ms/step - loss: 3.1741
Epoch 4/20
2/2 - 0s - 12ms/step - loss: 3.1421
Epoch 5/20
2/2 - 0s - 14ms/step - loss: 3.1098
Epoch 6/20
2/2 - 0s - 12ms/step - loss: 3.0741
Epoch 7/20
2/2 - 0s - 12ms/step - loss: 3.0484
Epoch 8/20
2/2 - 0s - 16ms/step - loss: 3.0143
Epoch 9/20
2/2 - 0s - 14ms/step - loss: 2.9879
Epoch 10/20
2/2 - 0s - 44ms/step - loss: 2.9612
Epoch 11/20
2/2 - 0s - 16ms/step - loss: 2.9309
Epoch 12/20
2/2 - 0s - 15ms/step - loss: 2.9059
Epoch 13/20
2/2 - 0s - 13ms/step - loss: 2.8828
Epoch 14/20
2/2 - 0s - 14ms/step - loss: 2.8544
Epoch 15/20
2/2 - 0s - 18ms/step - loss: 2.8254
Epoch 16/20
2/2 - 0s - 16ms/step - loss: 2.8104
Epoch 17/20
2/2 - 0s - 13ms/step - loss: 2.7852
Epoch 18/20
2/2 - 0s - 21ms/step - loss: 2.7569
Epoch 19/20
2/2 - 0s - 17ms/step - loss: 2.7431
Epoch 20/20
2/2 - 0s - 20ms/step - loss: 2.7189
2/2 - 1s - 369ms/step
[CV] END ..............................l1_reg=0.1, n_units=5; total time=   5.2s
Epoch 1/20
4/4 - 3s - 738ms/step - loss: 3.2764
Epoch 2/20
4/4 - 0s - 7ms/step - loss: 3.2083
Epoch 3/20
4/4 - 0s - 8ms/step - loss: 3.1567
Epoch 4/20
4/4 - 0s - 8ms/step - loss: 3.0986
Epoch 5/20
4/4 - 0s - 8ms/step - loss: 3.0466
Epoch 6/20
4/4 - 0s - 8ms/step - loss: 2.9987
Epoch 7/20
4/4 - 0s - 13ms/step - loss: 2.9469
Epoch 8/20
4/4 - 0s - 11ms/step - loss: 2.8918
Epoch 9/20
4/4 - 0s - 8ms/step - loss: 2.8486
Epoch 10/20
4/4 - 0s - 12ms/step - loss: 2.7994
Epoch 11/20
4/4 - 0s - 10ms/step - loss: 2.7635
Epoch 12/20
4/4 - 0s - 9ms/step - loss: 2.7172
Epoch 13/20
4/4 - 0s - 8ms/step - loss: 2.6800
Epoch 14/20
4/4 - 0s - 24ms/step - loss: 2.6421
Epoch 15/20
4/4 - 0s - 8ms/step - loss: 2.6153
Epoch 16/20
4/4 - 0s - 8ms/step - loss: 2.5776
Epoch 17/20
4/4 - 0s - 9ms/step - loss: 2.5505
Epoch 18/20
4/4 - 0s - 8ms/step - loss: 2.5269
Epoch 19/20
4/4 - 0s - 12ms/step - loss: 2.5013
Epoch 20/20
4/4 - 0s - 12ms/step - loss: 2.4725
2/2 - 1s - 409ms/step
[CV] END ..............................l1_reg=0.1, n_units=5; total time=   4.6s
Epoch 1/20
5/5 - 3s - 549ms/step - loss: 2.9676
Epoch 2/20
5/5 - 0s - 7ms/step - loss: 2.8853
Epoch 3/20
5/5 - 0s - 7ms/step - loss: 2.8154
Epoch 4/20
5/5 - 0s - 8ms/step - loss: 2.7360
Epoch 5/20
5/5 - 0s - 6ms/step - loss: 2.6750
Epoch 6/20
5/5 - 0s - 8ms/step - loss: 2.6097
Epoch 7/20
5/5 - 0s - 7ms/step - loss: 2.5456
Epoch 8/20
5/5 - 0s - 7ms/step - loss: 2.4904
Epoch 9/20
5/5 - 0s - 7ms/step - loss: 2.4352
Epoch 10/20
5/5 - 0s - 10ms/step - loss: 2.3826
Epoch 11/20
5/5 - 0s - 9ms/step - loss: 2.3400
Epoch 12/20
5/5 - 0s - 8ms/step - loss: 2.2867
Epoch 13/20
5/5 - 0s - 8ms/step - loss: 2.2463
Epoch 14/20
5/5 - 0s - 8ms/step - loss: 2.2087
Epoch 15/20
5/5 - 0s - 8ms/step - loss: 2.1688
Epoch 16/20
5/5 - 0s - 7ms/step - loss: 2.1377
Epoch 17/20
5/5 - 0s - 8ms/step - loss: 2.1044
Epoch 18/20
5/5 - 0s - 7ms/step - loss: 2.0691
Epoch 19/20
5/5 - 0s - 9ms/step - loss: 2.0449
Epoch 20/20
5/5 - 0s - 8ms/step - loss: 2.0133
2/2 - 1s - 459ms/step
[CV] END ..............................l1_reg=0.1, n_units=5; total time=   4.4s
Epoch 1/20
7/7 - 3s - 427ms/step - loss: 2.9203
Epoch 2/20
7/7 - 0s - 6ms/step - loss: 2.8128
Epoch 3/20
7/7 - 0s - 7ms/step - loss: 2.7008
Epoch 4/20
7/7 - 0s - 6ms/step - loss: 2.6153
Epoch 5/20
7/7 - 0s - 7ms/step - loss: 2.5244
Epoch 6/20
7/7 - 0s - 7ms/step - loss: 2.4422
Epoch 7/20
7/7 - 0s - 7ms/step - loss: 2.3713
Epoch 8/20
7/7 - 0s - 6ms/step - loss: 2.3070
Epoch 9/20
7/7 - 0s - 6ms/step - loss: 2.2439
Epoch 10/20
7/7 - 0s - 6ms/step - loss: 2.1929
Epoch 11/20
7/7 - 0s - 6ms/step - loss: 2.1430
Epoch 12/20
7/7 - 0s - 6ms/step - loss: 2.0991
Epoch 13/20
7/7 - 0s - 7ms/step - loss: 2.0609
Epoch 14/20
7/7 - 0s - 7ms/step - loss: 2.0228
Epoch 15/20
7/7 - 0s - 6ms/step - loss: 1.9803
Epoch 16/20
7/7 - 0s - 8ms/step - loss: 1.9480
Epoch 17/20
7/7 - 0s - 6ms/step - loss: 1.9164
Epoch 18/20
7/7 - 0s - 7ms/step - loss: 1.8893
Epoch 19/20
7/7 - 0s - 6ms/step - loss: 1.8602
Epoch 20/20
7/7 - 0s - 6ms/step - loss: 1.8285
2/2 - 1s - 473ms/step
[CV] END ..............................l1_reg=0.1, n_units=5; total time=   4.9s
Epoch 1/20
8/8 - 3s - 370ms/step - loss: 2.7735
Epoch 2/20
8/8 - 0s - 4ms/step - loss: 2.6450
Epoch 3/20
8/8 - 0s - 5ms/step - loss: 2.5225
Epoch 4/20
8/8 - 0s - 5ms/step - loss: 2.4210
Epoch 5/20
8/8 - 0s - 5ms/step - loss: 2.3160
Epoch 6/20
8/8 - 0s - 5ms/step - loss: 2.2221
Epoch 7/20
8/8 - 0s - 6ms/step - loss: 2.1419
Epoch 8/20
8/8 - 0s - 6ms/step - loss: 2.0639
Epoch 9/20
8/8 - 0s - 6ms/step - loss: 1.9975
Epoch 10/20
8/8 - 0s - 6ms/step - loss: 1.9379
Epoch 11/20
8/8 - 0s - 7ms/step - loss: 1.8845
Epoch 12/20
8/8 - 0s - 6ms/step - loss: 1.8330
Epoch 13/20
8/8 - 0s - 5ms/step - loss: 1.7922
Epoch 14/20
8/8 - 0s - 6ms/step - loss: 1.7464
Epoch 15/20
8/8 - 0s - 5ms/step - loss: 1.7160
Epoch 16/20
8/8 - 0s - 5ms/step - loss: 1.6825
Epoch 17/20
8/8 - 0s - 6ms/step - loss: 1.6483
Epoch 18/20
8/8 - 0s - 5ms/step - loss: 1.6230
Epoch 19/20
8/8 - 0s - 10ms/step - loss: 1.5983
Epoch 20/20
8/8 - 0s - 10ms/step - loss: 1.5789
2/2 - 1s - 415ms/step
[CV] END ..............................l1_reg=0.1, n_units=5; total time=   4.8s
Epoch 1/20
2/2 - 3s - 1s/step - loss: 3.9184
Epoch 2/20
2/2 - 0s - 14ms/step - loss: 3.8492
Epoch 3/20
2/2 - 0s - 15ms/step - loss: 3.7941
Epoch 4/20
2/2 - 0s - 13ms/step - loss: 3.7240
Epoch 5/20
2/2 - 0s - 12ms/step - loss: 3.6610
Epoch 6/20
2/2 - 0s - 17ms/step - loss: 3.6122
Epoch 7/20
2/2 - 0s - 14ms/step - loss: 3.5523
Epoch 8/20
2/2 - 0s - 14ms/step - loss: 3.4924
Epoch 9/20
2/2 - 0s - 13ms/step - loss: 3.4386
Epoch 10/20
2/2 - 0s - 20ms/step - loss: 3.3909
Epoch 11/20
2/2 - 0s - 14ms/step - loss: 3.3377
Epoch 12/20
2/2 - 0s - 17ms/step - loss: 3.2912
Epoch 13/20
2/2 - 0s - 18ms/step - loss: 3.2375
Epoch 14/20
2/2 - 0s - 23ms/step - loss: 3.1972
Epoch 15/20
2/2 - 0s - 17ms/step - loss: 3.1460
Epoch 16/20
2/2 - 0s - 14ms/step - loss: 3.1028
Epoch 17/20
2/2 - 0s - 19ms/step - loss: 3.0673
Epoch 18/20
2/2 - 0s - 19ms/step - loss: 3.0233
Epoch 19/20
2/2 - 0s - 17ms/step - loss: 2.9861
Epoch 20/20
2/2 - 0s - 22ms/step - loss: 2.9454
2/2 - 1s - 359ms/step
[CV] END .............................l1_reg=0.1, n_units=10; total time=   4.2s
Epoch 1/20
4/4 - 3s - 718ms/step - loss: 4.0567
Epoch 2/20
4/4 - 0s - 6ms/step - loss: 3.9170
Epoch 3/20
4/4 - 0s - 7ms/step - loss: 3.8093
Epoch 4/20
4/4 - 0s - 8ms/step - loss: 3.6904
Epoch 5/20
4/4 - 0s - 7ms/step - loss: 3.5830
Epoch 6/20
4/4 - 0s - 7ms/step - loss: 3.4824
Epoch 7/20
4/4 - 0s - 8ms/step - loss: 3.3850
Epoch 8/20
4/4 - 0s - 8ms/step - loss: 3.2916
Epoch 9/20
4/4 - 0s - 9ms/step - loss: 3.2035
Epoch 10/20
4/4 - 0s - 11ms/step - loss: 3.1264
Epoch 11/20
4/4 - 0s - 9ms/step - loss: 3.0546
Epoch 12/20
4/4 - 0s - 11ms/step - loss: 2.9847
Epoch 13/20
4/4 - 0s - 9ms/step - loss: 2.9314
Epoch 14/20
4/4 - 0s - 13ms/step - loss: 2.8790
Epoch 15/20
4/4 - 0s - 12ms/step - loss: 2.8349
Epoch 16/20
4/4 - 0s - 8ms/step - loss: 2.7903
Epoch 17/20
4/4 - 0s - 12ms/step - loss: 2.7529
Epoch 18/20
4/4 - 0s - 10ms/step - loss: 2.7196
Epoch 19/20
4/4 - 0s - 9ms/step - loss: 2.6790
Epoch 20/20
4/4 - 0s - 13ms/step - loss: 2.6400
2/2 - 1s - 385ms/step
[CV] END .............................l1_reg=0.1, n_units=10; total time=   4.4s
Epoch 1/20
5/5 - 3s - 536ms/step - loss: 3.7638
Epoch 2/20
5/5 - 0s - 6ms/step - loss: 3.5856
Epoch 3/20
5/5 - 0s - 7ms/step - loss: 3.4254
Epoch 4/20
5/5 - 0s - 14ms/step - loss: 3.2773
Epoch 5/20
5/5 - 0s - 12ms/step - loss: 3.1385
Epoch 6/20
5/5 - 0s - 8ms/step - loss: 3.0169
Epoch 7/20
5/5 - 0s - 6ms/step - loss: 2.8971
Epoch 8/20
5/5 - 0s - 7ms/step - loss: 2.7888
Epoch 9/20
5/5 - 0s - 7ms/step - loss: 2.6942
Epoch 10/20
5/5 - 0s - 7ms/step - loss: 2.6099
Epoch 11/20
5/5 - 0s - 9ms/step - loss: 2.5313
Epoch 12/20
5/5 - 0s - 11ms/step - loss: 2.4499
Epoch 13/20
5/5 - 0s - 9ms/step - loss: 2.3860
Epoch 14/20
5/5 - 0s - 8ms/step - loss: 2.3317
Epoch 15/20
5/5 - 0s - 9ms/step - loss: 2.2815
Epoch 16/20
5/5 - 0s - 9ms/step - loss: 2.2342
Epoch 17/20
5/5 - 0s - 9ms/step - loss: 2.1957
Epoch 18/20
5/5 - 0s - 9ms/step - loss: 2.1544
Epoch 19/20
5/5 - 0s - 11ms/step - loss: 2.1313
Epoch 20/20
5/5 - 0s - 11ms/step - loss: 2.1044
2/2 - 1s - 448ms/step
[CV] END .............................l1_reg=0.1, n_units=10; total time=   4.5s
Epoch 1/20
7/7 - 3s - 391ms/step - loss: 3.7167
Epoch 2/20
7/7 - 0s - 5ms/step - loss: 3.4727
Epoch 3/20
7/7 - 0s - 6ms/step - loss: 3.2695
Epoch 4/20
7/7 - 0s - 5ms/step - loss: 3.0889
Epoch 5/20
7/7 - 0s - 5ms/step - loss: 2.9362
Epoch 6/20
7/7 - 0s - 6ms/step - loss: 2.7936
Epoch 7/20
7/7 - 0s - 5ms/step - loss: 2.6678
Epoch 8/20
7/7 - 0s - 8ms/step - loss: 2.5496
Epoch 9/20
7/7 - 0s - 7ms/step - loss: 2.4508
Epoch 10/20
7/7 - 0s - 6ms/step - loss: 2.3541
Epoch 11/20
7/7 - 0s - 5ms/step - loss: 2.2691
Epoch 12/20
7/7 - 0s - 6ms/step - loss: 2.1996
Epoch 13/20
7/7 - 0s - 18ms/step - loss: 2.1543
Epoch 14/20
7/7 - 0s - 6ms/step - loss: 2.1152
Epoch 15/20
7/7 - 0s - 6ms/step - loss: 2.0748
Epoch 16/20
7/7 - 0s - 6ms/step - loss: 2.0405
Epoch 17/20
7/7 - 0s - 6ms/step - loss: 2.0092
Epoch 18/20
7/7 - 0s - 6ms/step - loss: 1.9754
Epoch 19/20
7/7 - 0s - 6ms/step - loss: 1.9421
Epoch 20/20
7/7 - 0s - 7ms/step - loss: 1.9103
2/2 - 1s - 472ms/step
[CV] END .............................l1_reg=0.1, n_units=10; total time=   4.6s
Epoch 1/20
8/8 - 3s - 341ms/step - loss: 3.5537
Epoch 2/20
8/8 - 0s - 4ms/step - loss: 3.2794
Epoch 3/20
8/8 - 0s - 4ms/step - loss: 3.0382
Epoch 4/20
8/8 - 0s - 5ms/step - loss: 2.8220
Epoch 5/20
8/8 - 0s - 5ms/step - loss: 2.6400
Epoch 6/20
8/8 - 0s - 6ms/step - loss: 2.4809
Epoch 7/20
8/8 - 0s - 5ms/step - loss: 2.3360
Epoch 8/20
8/8 - 0s - 5ms/step - loss: 2.2231
Epoch 9/20
8/8 - 0s - 5ms/step - loss: 2.1280
Epoch 10/20
8/8 - 0s - 5ms/step - loss: 2.0460
Epoch 11/20
8/8 - 0s - 5ms/step - loss: 1.9784
Epoch 12/20
8/8 - 0s - 11ms/step - loss: 1.9272
Epoch 13/20
8/8 - 0s - 8ms/step - loss: 1.8767
Epoch 14/20
8/8 - 0s - 5ms/step - loss: 1.8446
Epoch 15/20
8/8 - 0s - 6ms/step - loss: 1.8134
Epoch 16/20
8/8 - 0s - 6ms/step - loss: 1.7897
Epoch 17/20
8/8 - 0s - 8ms/step - loss: 1.7637
Epoch 18/20
8/8 - 0s - 8ms/step - loss: 1.7417
Epoch 19/20
8/8 - 0s - 8ms/step - loss: 1.7215
Epoch 20/20
8/8 - 0s - 8ms/step - loss: 1.7015
2/2 - 1s - 432ms/step
[CV] END .............................l1_reg=0.1, n_units=10; total time=   4.6s
Epoch 1/20
2/2 - 3s - 1s/step - loss: 3.8408
Epoch 2/20
2/2 - 0s - 12ms/step - loss: 3.7589
Epoch 3/20
2/2 - 0s - 14ms/step - loss: 3.6833
Epoch 4/20
2/2 - 0s - 13ms/step - loss: 3.6201
Epoch 5/20
2/2 - 0s - 13ms/step - loss: 3.5512
Epoch 6/20
2/2 - 0s - 13ms/step - loss: 3.5027
Epoch 7/20
2/2 - 0s - 14ms/step - loss: 3.4361
Epoch 8/20
2/2 - 0s - 14ms/step - loss: 3.3739
Epoch 9/20
2/2 - 0s - 14ms/step - loss: 3.3316
Epoch 10/20
2/2 - 0s - 12ms/step - loss: 3.2808
Epoch 11/20
2/2 - 0s - 17ms/step - loss: 3.2308
Epoch 12/20
2/2 - 0s - 19ms/step - loss: 3.1853
Epoch 13/20
2/2 - 0s - 44ms/step - loss: 3.1381
Epoch 14/20
2/2 - 0s - 19ms/step - loss: 3.0917
Epoch 15/20
2/2 - 0s - 16ms/step - loss: 3.0651
Epoch 16/20
2/2 - 0s - 16ms/step - loss: 3.0141
Epoch 17/20
2/2 - 0s - 19ms/step - loss: 2.9806
Epoch 18/20
2/2 - 0s - 22ms/step - loss: 2.9439
Epoch 19/20
2/2 - 0s - 17ms/step - loss: 2.9263
Epoch 20/20
2/2 - 0s - 23ms/step - loss: 2.8812
2/2 - 1s - 487ms/step
[CV] END .............................l1_reg=0.1, n_units=20; total time=   4.5s
Epoch 1/20
4/4 - 3s - 707ms/step - loss: 3.8113
Epoch 2/20
4/4 - 0s - 8ms/step - loss: 3.6611
Epoch 3/20
4/4 - 0s - 8ms/step - loss: 3.5337
Epoch 4/20
4/4 - 0s - 7ms/step - loss: 3.4280
Epoch 5/20
4/4 - 0s - 7ms/step - loss: 3.3403
Epoch 6/20
4/4 - 0s - 8ms/step - loss: 3.2401
Epoch 7/20
4/4 - 0s - 8ms/step - loss: 3.1654
Epoch 8/20
4/4 - 0s - 7ms/step - loss: 3.0924
Epoch 9/20
4/4 - 0s - 10ms/step - loss: 3.0080
Epoch 10/20
4/4 - 0s - 8ms/step - loss: 2.9347
Epoch 11/20
4/4 - 0s - 10ms/step - loss: 2.8846
Epoch 12/20
4/4 - 0s - 11ms/step - loss: 2.8189
Epoch 13/20
4/4 - 0s - 8ms/step - loss: 2.7697
Epoch 14/20
4/4 - 0s - 9ms/step - loss: 2.7231
Epoch 15/20
4/4 - 0s - 8ms/step - loss: 2.7057
Epoch 16/20
4/4 - 0s - 11ms/step - loss: 2.6814
Epoch 17/20
4/4 - 0s - 9ms/step - loss: 2.6488
Epoch 18/20
4/4 - 0s - 10ms/step - loss: 2.6293
Epoch 19/20
4/4 - 0s - 13ms/step - loss: 2.6097
Epoch 20/20
4/4 - 0s - 12ms/step - loss: 2.5887
2/2 - 1s - 398ms/step
[CV] END .............................l1_reg=0.1, n_units=20; total time=   4.4s
Epoch 1/20
5/5 - 3s - 608ms/step - loss: 3.4844
Epoch 2/20
5/5 - 0s - 7ms/step - loss: 3.2939
Epoch 3/20
5/5 - 0s - 7ms/step - loss: 3.1394
Epoch 4/20
5/5 - 0s - 6ms/step - loss: 2.9922
Epoch 5/20
5/5 - 0s - 6ms/step - loss: 2.8671
Epoch 6/20
5/5 - 0s - 7ms/step - loss: 2.7527
Epoch 7/20
5/5 - 0s - 6ms/step - loss: 2.6569
Epoch 8/20
5/5 - 0s - 7ms/step - loss: 2.5859
Epoch 9/20
5/5 - 0s - 7ms/step - loss: 2.5103
Epoch 10/20
5/5 - 0s - 6ms/step - loss: 2.4700
Epoch 11/20
5/5 - 0s - 9ms/step - loss: 2.4215
Epoch 12/20
5/5 - 0s - 9ms/step - loss: 2.3824
Epoch 13/20
5/5 - 0s - 10ms/step - loss: 2.3647
Epoch 14/20
5/5 - 0s - 10ms/step - loss: 2.3234
Epoch 15/20
5/5 - 0s - 8ms/step - loss: 2.2946
Epoch 16/20
5/5 - 0s - 10ms/step - loss: 2.2695
Epoch 17/20
5/5 - 0s - 9ms/step - loss: 2.2413
Epoch 18/20
5/5 - 0s - 8ms/step - loss: 2.2160
Epoch 19/20
5/5 - 0s - 7ms/step - loss: 2.1912
Epoch 20/20
5/5 - 0s - 9ms/step - loss: 2.1659
2/2 - 2s - 921ms/step
[CV] END .............................l1_reg=0.1, n_units=20; total time=   5.7s
Epoch 1/20
7/7 - 3s - 396ms/step - loss: 3.3925
Epoch 2/20
7/7 - 0s - 5ms/step - loss: 3.1325
Epoch 3/20
7/7 - 0s - 6ms/step - loss: 2.9313
Epoch 4/20
7/7 - 0s - 6ms/step - loss: 2.7916
Epoch 5/20
7/7 - 0s - 5ms/step - loss: 2.6561
Epoch 6/20
7/7 - 0s - 6ms/step - loss: 2.5399
Epoch 7/20
7/7 - 0s - 6ms/step - loss: 2.4420
Epoch 8/20
7/7 - 0s - 6ms/step - loss: 2.3803
Epoch 9/20
7/7 - 0s - 6ms/step - loss: 2.3152
Epoch 10/20
7/7 - 0s - 9ms/step - loss: 2.2687
Epoch 11/20
7/7 - 0s - 8ms/step - loss: 2.2395
Epoch 12/20
7/7 - 0s - 6ms/step - loss: 2.2074
Epoch 13/20
7/7 - 0s - 7ms/step - loss: 2.1648
Epoch 14/20
7/7 - 0s - 8ms/step - loss: 2.1357
Epoch 15/20
7/7 - 0s - 6ms/step - loss: 2.0921
Epoch 16/20
7/7 - 0s - 15ms/step - loss: 2.0596
Epoch 17/20
7/7 - 0s - 7ms/step - loss: 2.0448
Epoch 18/20
7/7 - 0s - 7ms/step - loss: 2.0154
Epoch 19/20
7/7 - 0s - 7ms/step - loss: 1.9809
Epoch 20/20
7/7 - 0s - 7ms/step - loss: 1.9523
2/2 - 1s - 478ms/step
[CV] END .............................l1_reg=0.1, n_units=20; total time=   4.7s
Epoch 1/20
8/8 - 3s - 352ms/step - loss: 3.2137
Epoch 2/20
8/8 - 0s - 10ms/step - loss: 2.9192
Epoch 3/20
8/8 - 0s - 4ms/step - loss: 2.7076
Epoch 4/20
8/8 - 0s - 5ms/step - loss: 2.5108
Epoch 5/20
8/8 - 0s - 5ms/step - loss: 2.4031
Epoch 6/20
8/8 - 0s - 5ms/step - loss: 2.2868
Epoch 7/20
8/8 - 0s - 6ms/step - loss: 2.2198
Epoch 8/20
8/8 - 0s - 6ms/step - loss: 2.1639
Epoch 9/20
8/8 - 0s - 6ms/step - loss: 2.1179
Epoch 10/20
8/8 - 0s - 8ms/step - loss: 2.0735
Epoch 11/20
8/8 - 0s - 7ms/step - loss: 2.0317
Epoch 12/20
8/8 - 0s - 6ms/step - loss: 1.9905
Epoch 13/20
8/8 - 0s - 7ms/step - loss: 1.9567
Epoch 14/20
8/8 - 0s - 7ms/step - loss: 1.9118
Epoch 15/20
8/8 - 0s - 5ms/step - loss: 1.8737
Epoch 16/20
8/8 - 0s - 7ms/step - loss: 1.8347
Epoch 17/20
8/8 - 0s - 6ms/step - loss: 1.7971
Epoch 18/20
8/8 - 0s - 8ms/step - loss: 1.7596
Epoch 19/20
8/8 - 0s - 8ms/step - loss: 1.7261
Epoch 20/20
8/8 - 0s - 7ms/step - loss: 1.6901
2/2 - 1s - 453ms/step
[CV] END .............................l1_reg=0.1, n_units=20; total time=   4.8s
Epoch 1/20
10/10 - 3s - 285ms/step - loss: 1.8129
Epoch 2/20
10/10 - 0s - 4ms/step - loss: 1.5277
Epoch 3/20
10/10 - 0s - 4ms/step - loss: 1.3087
Epoch 4/20
10/10 - 0s - 4ms/step - loss: 1.1722
Epoch 5/20
10/10 - 0s - 5ms/step - loss: 1.1282
Epoch 6/20
10/10 - 0s - 4ms/step - loss: 1.0871
Epoch 7/20
10/10 - 0s - 5ms/step - loss: 1.0770
Epoch 8/20
10/10 - 0s - 6ms/step - loss: 1.0674
Epoch 9/20
10/10 - 0s - 5ms/step - loss: 1.0588
Epoch 10/20
10/10 - 0s - 5ms/step - loss: 1.0539
Epoch 11/20
10/10 - 0s - 5ms/step - loss: 1.0457
Epoch 12/20
10/10 - 0s - 5ms/step - loss: 1.0374
Epoch 13/20
10/10 - 0s - 6ms/step - loss: 1.0305
Epoch 14/20
10/10 - 0s - 6ms/step - loss: 1.0309
Epoch 15/20
10/10 - 0s - 8ms/step - loss: 1.0300
Epoch 16/20
10/10 - 0s - 7ms/step - loss: 1.0304
Epoch 17/20
10/10 - 0s - 7ms/step - loss: 1.0274
Epoch 18/20
10/10 - 0s - 6ms/step - loss: 1.0198
Epoch 19/20
10/10 - 0s - 6ms/step - loss: 1.0190
Epoch 20/20
10/10 - 0s - 7ms/step - loss: 1.0176
Best: 0.034584 using {'l1_reg': 0.01, 'n_units': 20}
nan (nan) with {'l1_reg': 0, 'n_units': 5}
nan (nan) with {'l1_reg': 0, 'n_units': 10}
nan (nan) with {'l1_reg': 0, 'n_units': 20}
-0.262233 (0.176903) with {'l1_reg': 0.001, 'n_units': 5}
-0.134744 (0.196529) with {'l1_reg': 0.001, 'n_units': 10}
0.031554 (0.046699) with {'l1_reg': 0.001, 'n_units': 20}
-0.270691 (0.186918) with {'l1_reg': 0.01, 'n_units': 5}
-0.117948 (0.192042) with {'l1_reg': 0.01, 'n_units': 10}
0.034584 (0.033399) with {'l1_reg': 0.01, 'n_units': 20}
-0.288396 (0.182568) with {'l1_reg': 0.1, 'n_units': 5}
-0.150610 (0.220931) with {'l1_reg': 0.1, 'n_units': 10}
0.011222 (0.033038) with {'l1_reg': 0.1, 'n_units': 20}
Performing cross-validation. Model: lstm
Fitting 5 folds for each of 12 candidates, totalling 60 fits
Epoch 1/20
[CV] END ................................l1_reg=0, n_units=5; total time=   0.3s
Epoch 1/20
[CV] END ................................l1_reg=0, n_units=5; total time=   0.2s
Epoch 1/20
[CV] END ................................l1_reg=0, n_units=5; total time=   0.2s
Epoch 1/20
[CV] END ................................l1_reg=0, n_units=5; total time=   0.2s
Epoch 1/20
[CV] END ................................l1_reg=0, n_units=5; total time=   0.2s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=10; total time=   0.1s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=10; total time=   0.1s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=10; total time=   0.2s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=10; total time=   0.2s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=10; total time=   0.1s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=20; total time=   0.1s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=20; total time=   0.1s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=20; total time=   0.1s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=20; total time=   0.1s
Epoch 1/20
[CV] END ...............................l1_reg=0, n_units=20; total time=   0.3s
Epoch 1/20
2/2 - 3s - 1s/step - loss: 1.7034
Epoch 2/20
2/2 - 0s - 22ms/step - loss: 1.6871
Epoch 3/20
2/2 - 0s - 19ms/step - loss: 1.6720
Epoch 4/20
2/2 - 0s - 19ms/step - loss: 1.6549
Epoch 5/20
2/2 - 0s - 19ms/step - loss: 1.6439
Epoch 6/20
2/2 - 0s - 33ms/step - loss: 1.6314
Epoch 7/20
2/2 - 0s - 23ms/step - loss: 1.6226
Epoch 8/20
2/2 - 0s - 27ms/step - loss: 1.6064
Epoch 9/20
2/2 - 0s - 57ms/step - loss: 1.5927
Epoch 10/20
2/2 - 0s - 21ms/step - loss: 1.5849
Epoch 11/20
2/2 - 0s - 19ms/step - loss: 1.5685
Epoch 12/20
2/2 - 0s - 24ms/step - loss: 1.5618
Epoch 13/20
2/2 - 0s - 31ms/step - loss: 1.5492
Epoch 14/20
2/2 - 0s - 22ms/step - loss: 1.5365
Epoch 15/20
2/2 - 0s - 31ms/step - loss: 1.5296
Epoch 16/20
2/2 - 0s - 26ms/step - loss: 1.5177
Epoch 17/20
2/2 - 0s - 23ms/step - loss: 1.5087
Epoch 18/20
2/2 - 0s - 21ms/step - loss: 1.4972
Epoch 19/20
2/2 - 0s - 23ms/step - loss: 1.4877
Epoch 20/20
2/2 - 0s - 23ms/step - loss: 1.4768
2/2 - 1s - 547ms/step
[CV] END ............................l1_reg=0.001, n_units=5; total time=   4.7s
Epoch 1/20
4/4 - 3s - 643ms/step - loss: 1.7078
Epoch 2/20
4/4 - 0s - 8ms/step - loss: 1.6801
Epoch 3/20
4/4 - 0s - 11ms/step - loss: 1.6587
Epoch 4/20
4/4 - 0s - 11ms/step - loss: 1.6368
Epoch 5/20
4/4 - 0s - 10ms/step - loss: 1.6188
Epoch 6/20
4/4 - 0s - 10ms/step - loss: 1.6001
Epoch 7/20
4/4 - 0s - 12ms/step - loss: 1.5848
Epoch 8/20
4/4 - 0s - 11ms/step - loss: 1.5675
Epoch 9/20
4/4 - 0s - 11ms/step - loss: 1.5498
Epoch 10/20
4/4 - 0s - 12ms/step - loss: 1.5350
Epoch 11/20
4/4 - 0s - 11ms/step - loss: 1.5203
Epoch 12/20
4/4 - 0s - 9ms/step - loss: 1.5066
Epoch 13/20
4/4 - 0s - 10ms/step - loss: 1.4994
Epoch 14/20
4/4 - 0s - 15ms/step - loss: 1.4899
Epoch 15/20
4/4 - 0s - 13ms/step - loss: 1.4803
Epoch 16/20
4/4 - 0s - 13ms/step - loss: 1.4743
Epoch 17/20
4/4 - 0s - 17ms/step - loss: 1.4606
Epoch 18/20
4/4 - 0s - 14ms/step - loss: 1.4509
Epoch 19/20
4/4 - 0s - 12ms/step - loss: 1.4405
Epoch 20/20
4/4 - 0s - 11ms/step - loss: 1.4353
2/2 - 1s - 419ms/step
[CV] END ............................l1_reg=0.001, n_units=5; total time=   4.4s
Epoch 1/20
5/5 - 2s - 458ms/step - loss: 1.4463
Epoch 2/20
5/5 - 0s - 6ms/step - loss: 1.4093
Epoch 3/20
5/5 - 0s - 6ms/step - loss: 1.3830
Epoch 4/20
5/5 - 0s - 7ms/step - loss: 1.3516
Epoch 5/20
5/5 - 0s - 8ms/step - loss: 1.3260
Epoch 6/20
5/5 - 0s - 8ms/step - loss: 1.2969
Epoch 7/20
5/5 - 0s - 9ms/step - loss: 1.2748
Epoch 8/20
5/5 - 0s - 11ms/step - loss: 1.2484
Epoch 9/20
5/5 - 0s - 10ms/step - loss: 1.2301
Epoch 10/20
5/5 - 0s - 10ms/step - loss: 1.2146
Epoch 11/20
5/5 - 0s - 9ms/step - loss: 1.1955
Epoch 12/20
5/5 - 0s - 15ms/step - loss: 1.1819
Epoch 13/20
5/5 - 0s - 8ms/step - loss: 1.1717
Epoch 14/20
5/5 - 0s - 11ms/step - loss: 1.1637
Epoch 15/20
5/5 - 0s - 11ms/step - loss: 1.1567
Epoch 16/20
5/5 - 0s - 9ms/step - loss: 1.1533
Epoch 17/20
5/5 - 0s - 8ms/step - loss: 1.1473
Epoch 18/20
5/5 - 0s - 10ms/step - loss: 1.1443
Epoch 19/20
5/5 - 0s - 11ms/step - loss: 1.1443
Epoch 20/20
5/5 - 0s - 9ms/step - loss: 1.1446
2/2 - 1s - 333ms/step
[CV] END ............................l1_reg=0.001, n_units=5; total time=   3.9s
Epoch 1/20
7/7 - 2s - 336ms/step - loss: 1.3617
Epoch 2/20
7/7 - 0s - 4ms/step - loss: 1.3134
Epoch 3/20
7/7 - 0s - 5ms/step - loss: 1.2709
Epoch 4/20
7/7 - 0s - 5ms/step - loss: 1.2347
Epoch 5/20
7/7 - 0s - 5ms/step - loss: 1.1973
Epoch 6/20
7/7 - 0s - 5ms/step - loss: 1.1664
Epoch 7/20
7/7 - 0s - 8ms/step - loss: 1.1427
Epoch 8/20
7/7 - 0s - 11ms/step - loss: 1.1281
Epoch 9/20
7/7 - 0s - 15ms/step - loss: 1.1156
Epoch 10/20
7/7 - 0s - 5ms/step - loss: 1.1021
Epoch 11/20
7/7 - 0s - 7ms/step - loss: 1.0901
Epoch 12/20
7/7 - 0s - 8ms/step - loss: 1.0863
Epoch 13/20
7/7 - 0s - 6ms/step - loss: 1.0783
Epoch 14/20
7/7 - 0s - 7ms/step - loss: 1.0775
Epoch 15/20
7/7 - 0s - 7ms/step - loss: 1.0769
Epoch 16/20
7/7 - 0s - 6ms/step - loss: 1.0737
Epoch 17/20
7/7 - 0s - 11ms/step - loss: 1.0720
Epoch 18/20
7/7 - 0s - 6ms/step - loss: 1.0716
Epoch 19/20
7/7 - 0s - 7ms/step - loss: 1.0693
Epoch 20/20
7/7 - 0s - 8ms/step - loss: 1.0674
2/2 - 1s - 440ms/step
[CV] END ............................l1_reg=0.001, n_units=5; total time=   4.3s
Epoch 1/20
8/8 - 2s - 291ms/step - loss: 1.2098
Epoch 2/20
8/8 - 0s - 5ms/step - loss: 1.1567
Epoch 3/20
8/8 - 0s - 5ms/step - loss: 1.1053
Epoch 4/20
8/8 - 0s - 5ms/step - loss: 1.0706
Epoch 5/20
8/8 - 0s - 5ms/step - loss: 1.0287
Epoch 6/20
8/8 - 0s - 5ms/step - loss: 0.9959
Epoch 7/20
8/8 - 0s - 6ms/step - loss: 0.9712
Epoch 8/20
8/8 - 0s - 6ms/step - loss: 0.9573
Epoch 9/20
8/8 - 0s - 6ms/step - loss: 0.9499
Epoch 10/20
8/8 - 0s - 5ms/step - loss: 0.9411
Epoch 11/20
8/8 - 0s - 7ms/step - loss: 0.9395
Epoch 12/20
8/8 - 0s - 6ms/step - loss: 0.9380
Epoch 13/20
8/8 - 0s - 6ms/step - loss: 0.9388
Epoch 14/20
8/8 - 0s - 5ms/step - loss: 0.9383
Epoch 15/20
8/8 - 0s - 7ms/step - loss: 0.9386
Epoch 16/20
8/8 - 0s - 8ms/step - loss: 0.9370
Epoch 17/20
8/8 - 0s - 8ms/step - loss: 0.9372
Epoch 18/20
8/8 - 0s - 8ms/step - loss: 0.9375
Epoch 19/20
8/8 - 0s - 7ms/step - loss: 0.9363
Epoch 20/20
8/8 - 0s - 7ms/step - loss: 0.9366
2/2 - 1s - 300ms/step
[CV] END ............................l1_reg=0.001, n_units=5; total time=   4.0s
Epoch 1/20
2/2 - 2s - 1s/step - loss: 1.5003
Epoch 2/20
2/2 - 0s - 13ms/step - loss: 1.4797
Epoch 3/20
2/2 - 0s - 14ms/step - loss: 1.4712
Epoch 4/20
2/2 - 0s - 12ms/step - loss: 1.4575
Epoch 5/20
2/2 - 0s - 14ms/step - loss: 1.4416
Epoch 6/20
2/2 - 0s - 13ms/step - loss: 1.4267
Epoch 7/20
2/2 - 0s - 15ms/step - loss: 1.4162
Epoch 8/20
2/2 - 0s - 13ms/step - loss: 1.3991
Epoch 9/20
2/2 - 0s - 14ms/step - loss: 1.3873
Epoch 10/20
2/2 - 0s - 13ms/step - loss: 1.3733
Epoch 11/20
2/2 - 0s - 14ms/step - loss: 1.3596
Epoch 12/20
2/2 - 0s - 14ms/step - loss: 1.3461
Epoch 13/20
2/2 - 0s - 14ms/step - loss: 1.3402
Epoch 14/20
2/2 - 0s - 20ms/step - loss: 1.3235
Epoch 15/20
2/2 - 0s - 14ms/step - loss: 1.3244
Epoch 16/20
2/2 - 0s - 20ms/step - loss: 1.3082
Epoch 17/20
2/2 - 0s - 15ms/step - loss: 1.3014
Epoch 18/20
2/2 - 0s - 17ms/step - loss: 1.2933
Epoch 19/20
2/2 - 0s - 15ms/step - loss: 1.2937
Epoch 20/20
2/2 - 0s - 20ms/step - loss: 1.2857
2/2 - 1s - 438ms/step
[CV] END ...........................l1_reg=0.001, n_units=10; total time=   3.6s
Epoch 1/20
4/4 - 2s - 554ms/step - loss: 1.5579
Epoch 2/20
4/4 - 0s - 7ms/step - loss: 1.5395
Epoch 3/20
4/4 - 0s - 9ms/step - loss: 1.5242
Epoch 4/20
4/4 - 0s - 8ms/step - loss: 1.5099
Epoch 5/20
4/4 - 0s - 7ms/step - loss: 1.4946
Epoch 6/20
4/4 - 0s - 7ms/step - loss: 1.4860
Epoch 7/20
4/4 - 0s - 8ms/step - loss: 1.4814
Epoch 8/20
4/4 - 0s - 9ms/step - loss: 1.4697
Epoch 9/20
4/4 - 0s - 11ms/step - loss: 1.4574
Epoch 10/20
4/4 - 0s - 9ms/step - loss: 1.4497
Epoch 11/20
4/4 - 0s - 8ms/step - loss: 1.4389
Epoch 12/20
4/4 - 0s - 9ms/step - loss: 1.4303
Epoch 13/20
4/4 - 0s - 8ms/step - loss: 1.4227
Epoch 14/20
4/4 - 0s - 10ms/step - loss: 1.4227
Epoch 15/20
4/4 - 0s - 7ms/step - loss: 1.4131
Epoch 16/20
4/4 - 0s - 11ms/step - loss: 1.4053
Epoch 17/20
4/4 - 0s - 8ms/step - loss: 1.4012
Epoch 18/20
4/4 - 0s - 10ms/step - loss: 1.3999
Epoch 19/20
4/4 - 0s - 11ms/step - loss: 1.3991
Epoch 20/20
4/4 - 0s - 11ms/step - loss: 1.3990
2/2 - 1s - 296ms/step
[CV] END ...........................l1_reg=0.001, n_units=10; total time=   3.6s
Epoch 1/20
5/5 - 2s - 405ms/step - loss: 1.3356
Epoch 2/20
5/5 - 0s - 7ms/step - loss: 1.2934
Epoch 3/20
5/5 - 0s - 8ms/step - loss: 1.2606
Epoch 4/20
5/5 - 0s - 7ms/step - loss: 1.2330
Epoch 5/20
5/5 - 0s - 7ms/step - loss: 1.2100
Epoch 6/20
5/5 - 0s - 8ms/step - loss: 1.1914
Epoch 7/20
5/5 - 0s - 7ms/step - loss: 1.1924
Epoch 8/20
5/5 - 0s - 8ms/step - loss: 1.1703
Epoch 9/20
5/5 - 0s - 9ms/step - loss: 1.1688
Epoch 10/20
5/5 - 0s - 7ms/step - loss: 1.1692
Epoch 11/20
5/5 - 0s - 8ms/step - loss: 1.1671
Epoch 12/20
5/5 - 0s - 9ms/step - loss: 1.1642
Epoch 13/20
5/5 - 0s - 8ms/step - loss: 1.1630
Epoch 14/20
5/5 - 0s - 7ms/step - loss: 1.1624
Epoch 15/20
5/5 - 0s - 8ms/step - loss: 1.1610
Epoch 16/20
5/5 - 0s - 10ms/step - loss: 1.1620
Epoch 17/20
5/5 - 0s - 12ms/step - loss: 1.1599
Epoch 18/20
5/5 - 0s - 10ms/step - loss: 1.1593
Epoch 19/20
5/5 - 0s - 10ms/step - loss: 1.1584
Epoch 20/20
5/5 - 0s - 9ms/step - loss: 1.1575
2/2 - 1s - 390ms/step
[CV] END ...........................l1_reg=0.001, n_units=10; total time=   3.7s
Epoch 1/20
7/7 - 2s - 335ms/step - loss: 1.2318
Epoch 2/20
7/7 - 0s - 5ms/step - loss: 1.1887
Epoch 3/20
7/7 - 0s - 5ms/step - loss: 1.1542
Epoch 4/20
7/7 - 0s - 5ms/step - loss: 1.1192
Epoch 5/20
7/7 - 0s - 5ms/step - loss: 1.1011
Epoch 6/20
7/7 - 0s - 5ms/step - loss: 1.0929
Epoch 7/20
7/7 - 0s - 5ms/step - loss: 1.0884
Epoch 8/20
7/7 - 0s - 5ms/step - loss: 1.0838
Epoch 9/20
7/7 - 0s - 5ms/step - loss: 1.0841
Epoch 10/20
7/7 - 0s - 7ms/step - loss: 1.0826
Epoch 11/20
7/7 - 0s - 8ms/step - loss: 1.0821
Epoch 12/20
7/7 - 0s - 8ms/step - loss: 1.0844
Epoch 13/20
7/7 - 0s - 6ms/step - loss: 1.0848
Epoch 14/20
7/7 - 0s - 7ms/step - loss: 1.0824
Epoch 15/20
7/7 - 0s - 6ms/step - loss: 1.0825
Epoch 16/20
7/7 - 0s - 6ms/step - loss: 1.0816
Epoch 17/20
7/7 - 0s - 7ms/step - loss: 1.0822
Epoch 18/20
7/7 - 0s - 8ms/step - loss: 1.0831
Epoch 19/20
7/7 - 0s - 9ms/step - loss: 1.0804
Epoch 20/20
7/7 - 0s - 8ms/step - loss: 1.0791
2/2 - 1s - 287ms/step
[CV] END ...........................l1_reg=0.001, n_units=10; total time=   3.9s
Epoch 1/20
8/8 - 2s - 269ms/step - loss: 1.0898
Epoch 2/20
8/8 - 0s - 5ms/step - loss: 1.0380
Epoch 3/20
8/8 - 0s - 5ms/step - loss: 1.0044
Epoch 4/20
8/8 - 0s - 5ms/step - loss: 0.9710
Epoch 5/20
8/8 - 0s - 5ms/step - loss: 0.9654
Epoch 6/20
8/8 - 0s - 6ms/step - loss: 0.9580
Epoch 7/20
8/8 - 0s - 5ms/step - loss: 0.9539
Epoch 8/20
8/8 - 0s - 5ms/step - loss: 0.9533
Epoch 9/20
8/8 - 0s - 5ms/step - loss: 0.9524
Epoch 10/20
8/8 - 0s - 6ms/step - loss: 0.9525
Epoch 11/20
8/8 - 0s - 6ms/step - loss: 0.9514
Epoch 12/20
8/8 - 0s - 6ms/step - loss: 0.9523
Epoch 13/20
8/8 - 0s - 6ms/step - loss: 0.9512
Epoch 14/20
8/8 - 0s - 5ms/step - loss: 0.9515
Epoch 15/20
8/8 - 0s - 6ms/step - loss: 0.9532
Epoch 16/20
8/8 - 0s - 7ms/step - loss: 0.9502
Epoch 17/20
8/8 - 0s - 7ms/step - loss: 0.9511
Epoch 18/20
8/8 - 0s - 6ms/step - loss: 0.9528
Epoch 19/20
8/8 - 0s - 6ms/step - loss: 0.9496
Epoch 20/20
8/8 - 0s - 7ms/step - loss: 0.9490
2/2 - 1s - 364ms/step
[CV] END ...........................l1_reg=0.001, n_units=10; total time=   3.8s
Epoch 1/20
2/2 - 3s - 1s/step - loss: 1.5470
Epoch 2/20
2/2 - 0s - 13ms/step - loss: 1.5298
Epoch 3/20
2/2 - 0s - 14ms/step - loss: 1.5050
Epoch 4/20
2/2 - 0s - 12ms/step - loss: 1.4840
Epoch 5/20
2/2 - 0s - 15ms/step - loss: 1.4720
Epoch 6/20
2/2 - 0s - 14ms/step - loss: 1.4572
Epoch 7/20
2/2 - 0s - 15ms/step - loss: 1.4371
Epoch 8/20
2/2 - 0s - 14ms/step - loss: 1.4173
Epoch 9/20
2/2 - 0s - 15ms/step - loss: 1.4002
Epoch 10/20
2/2 - 0s - 14ms/step - loss: 1.3896
Epoch 11/20
2/2 - 0s - 14ms/step - loss: 1.3789
Epoch 12/20
2/2 - 0s - 15ms/step - loss: 1.3691
Epoch 13/20
2/2 - 0s - 16ms/step - loss: 1.3541
Epoch 14/20
2/2 - 0s - 17ms/step - loss: 1.3460
Epoch 15/20
2/2 - 0s - 17ms/step - loss: 1.3383
Epoch 16/20
2/2 - 0s - 17ms/step - loss: 1.3235
Epoch 17/20
2/2 - 0s - 21ms/step - loss: 1.3182
Epoch 18/20
2/2 - 0s - 21ms/step - loss: 1.3101
Epoch 19/20
2/2 - 0s - 20ms/step - loss: 1.3030
Epoch 20/20
2/2 - 0s - 20ms/step - loss: 1.3015
2/2 - 1s - 515ms/step
[CV] END ...........................l1_reg=0.001, n_units=20; total time=   4.3s
Epoch 1/20
4/4 - 3s - 697ms/step - loss: 1.5570
Epoch 2/20
4/4 - 0s - 7ms/step - loss: 1.5311
Epoch 3/20
4/4 - 0s - 10ms/step - loss: 1.5115
Epoch 4/20
4/4 - 0s - 7ms/step - loss: 1.5104
Epoch 5/20
4/4 - 0s - 9ms/step - loss: 1.5046
Epoch 6/20
4/4 - 0s - 8ms/step - loss: 1.4951
Epoch 7/20
4/4 - 0s - 8ms/step - loss: 1.4885
Epoch 8/20
4/4 - 0s - 8ms/step - loss: 1.4793
Epoch 9/20
4/4 - 0s - 9ms/step - loss: 1.4767
Epoch 10/20
4/4 - 0s - 8ms/step - loss: 1.4746
Epoch 11/20
4/4 - 0s - 8ms/step - loss: 1.4683
Epoch 12/20
4/4 - 0s - 8ms/step - loss: 1.4681
Epoch 13/20
4/4 - 0s - 10ms/step - loss: 1.4582
Epoch 14/20
4/4 - 0s - 10ms/step - loss: 1.4491
Epoch 15/20
4/4 - 0s - 11ms/step - loss: 1.4553
Epoch 16/20
4/4 - 0s - 9ms/step - loss: 1.4506
Epoch 17/20
4/4 - 0s - 9ms/step - loss: 1.4449
Epoch 18/20
4/4 - 0s - 12ms/step - loss: 1.4405
Epoch 19/20
4/4 - 0s - 10ms/step - loss: 1.4372
Epoch 20/20
4/4 - 0s - 10ms/step - loss: 1.4404
2/2 - 1s - 331ms/step
[CV] END ...........................l1_reg=0.001, n_units=20; total time=   4.3s
Epoch 1/20
5/5 - 2s - 430ms/step - loss: 1.2707
Epoch 2/20
5/5 - 0s - 6ms/step - loss: 1.2243
Epoch 3/20
5/5 - 0s - 8ms/step - loss: 1.1884
Epoch 4/20
5/5 - 0s - 6ms/step - loss: 1.1797
Epoch 5/20
5/5 - 0s - 7ms/step - loss: 1.1718
Epoch 6/20
5/5 - 0s - 7ms/step - loss: 1.1710
Epoch 7/20
5/5 - 0s - 9ms/step - loss: 1.1698
Epoch 8/20
5/5 - 0s - 8ms/step - loss: 1.1754
Epoch 9/20
5/5 - 0s - 8ms/step - loss: 1.1732
Epoch 10/20
5/5 - 0s - 9ms/step - loss: 1.1695
Epoch 11/20
5/5 - 0s - 8ms/step - loss: 1.1692
Epoch 12/20
5/5 - 0s - 10ms/step - loss: 1.1681
Epoch 13/20
5/5 - 0s - 8ms/step - loss: 1.1680
Epoch 14/20
5/5 - 0s - 8ms/step - loss: 1.1677
Epoch 15/20
5/5 - 0s - 7ms/step - loss: 1.1675
Epoch 16/20
5/5 - 0s - 10ms/step - loss: 1.1665
Epoch 17/20
5/5 - 0s - 9ms/step - loss: 1.1663
Epoch 18/20
5/5 - 0s - 8ms/step - loss: 1.1659
Epoch 19/20
5/5 - 0s - 8ms/step - loss: 1.1651
Epoch 20/20
5/5 - 0s - 8ms/step - loss: 1.1633
2/2 - 1s - 269ms/step
[CV] END ...........................l1_reg=0.001, n_units=20; total time=   3.5s
Epoch 1/20
7/7 - 2s - 302ms/step - loss: 1.2195
Epoch 2/20
7/7 - 0s - 5ms/step - loss: 1.1581
Epoch 3/20
7/7 - 0s - 5ms/step - loss: 1.1308
Epoch 4/20
7/7 - 0s - 4ms/step - loss: 1.1173
Epoch 5/20
7/7 - 0s - 5ms/step - loss: 1.1109
Epoch 6/20
7/7 - 0s - 5ms/step - loss: 1.1060
Epoch 7/20
7/7 - 0s - 7ms/step - loss: 1.1061
Epoch 8/20
7/7 - 0s - 5ms/step - loss: 1.1063
Epoch 9/20
7/7 - 0s - 5ms/step - loss: 1.1122
Epoch 10/20
7/7 - 0s - 5ms/step - loss: 1.1157
Epoch 11/20
7/7 - 0s - 5ms/step - loss: 1.1116
Epoch 12/20
7/7 - 0s - 6ms/step - loss: 1.0985
Epoch 13/20
7/7 - 0s - 5ms/step - loss: 1.0934
Epoch 14/20
7/7 - 0s - 6ms/step - loss: 1.1022
Epoch 15/20
7/7 - 0s - 6ms/step - loss: 1.1069
Epoch 16/20
7/7 - 0s - 8ms/step - loss: 1.1048
Epoch 17/20
7/7 - 0s - 7ms/step - loss: 1.1055
Epoch 18/20
7/7 - 0s - 7ms/step - loss: 1.0950
Epoch 19/20
7/7 - 0s - 7ms/step - loss: 1.0902
Epoch 20/20
7/7 - 0s - 7ms/step - loss: 1.0989
2/2 - 1s - 278ms/step
[CV] END ...........................l1_reg=0.001, n_units=20; total time=   3.5s
Epoch 1/20
8/8 - 3s - 353ms/step - loss: 1.0575
Epoch 2/20
8/8 - 0s - 7ms/step - loss: 1.0191
Epoch 3/20
8/8 - 0s - 6ms/step - loss: 0.9862
Epoch 4/20
8/8 - 0s - 7ms/step - loss: 0.9815
Epoch 5/20
8/8 - 0s - 6ms/step - loss: 0.9768
Epoch 6/20
8/8 - 0s - 7ms/step - loss: 0.9780
Epoch 7/20
8/8 - 0s - 8ms/step - loss: 0.9753
Epoch 8/20
8/8 - 0s - 6ms/step - loss: 0.9744
Epoch 9/20
8/8 - 0s - 6ms/step - loss: 0.9751
Epoch 10/20
8/8 - 0s - 7ms/step - loss: 0.9705
Epoch 11/20
8/8 - 0s - 6ms/step - loss: 0.9720
Epoch 12/20
8/8 - 0s - 6ms/step - loss: 0.9696
Epoch 13/20
8/8 - 0s - 9ms/step - loss: 0.9701
Epoch 14/20
8/8 - 0s - 8ms/step - loss: 0.9682
Epoch 15/20
8/8 - 0s - 10ms/step - loss: 0.9684
Epoch 16/20
8/8 - 0s - 9ms/step - loss: 0.9689
Epoch 17/20
8/8 - 0s - 10ms/step - loss: 0.9733
Epoch 18/20
8/8 - 0s - 20ms/step - loss: 0.9658
Epoch 19/20
8/8 - 0s - 9ms/step - loss: 0.9650
Epoch 20/20
8/8 - 0s - 9ms/step - loss: 0.9628
2/2 - 1s - 459ms/step
[CV] END ...........................l1_reg=0.001, n_units=20; total time=   5.1s
Epoch 1/20
2/2 - 2s - 1s/step - loss: 1.7728
Epoch 2/20
2/2 - 0s - 13ms/step - loss: 1.7588
Epoch 3/20
2/2 - 0s - 15ms/step - loss: 1.7458
Epoch 4/20
2/2 - 0s - 16ms/step - loss: 1.7320
Epoch 5/20
2/2 - 0s - 14ms/step - loss: 1.7190
Epoch 6/20
2/2 - 0s - 14ms/step - loss: 1.7039
Epoch 7/20
2/2 - 0s - 13ms/step - loss: 1.6978
Epoch 8/20
2/2 - 0s - 14ms/step - loss: 1.6851
Epoch 9/20
2/2 - 0s - 15ms/step - loss: 1.6719
Epoch 10/20
2/2 - 0s - 15ms/step - loss: 1.6621
Epoch 11/20
2/2 - 0s - 17ms/step - loss: 1.6542
Epoch 12/20
2/2 - 0s - 16ms/step - loss: 1.6396
Epoch 13/20
2/2 - 0s - 19ms/step - loss: 1.6284
Epoch 14/20
2/2 - 0s - 23ms/step - loss: 1.6236
Epoch 15/20
2/2 - 0s - 21ms/step - loss: 1.6109
Epoch 16/20
2/2 - 0s - 16ms/step - loss: 1.6005
Epoch 17/20
2/2 - 0s - 16ms/step - loss: 1.5913
Epoch 18/20
2/2 - 0s - 15ms/step - loss: 1.5780
Epoch 19/20
2/2 - 0s - 18ms/step - loss: 1.5728
Epoch 20/20
2/2 - 0s - 19ms/step - loss: 1.5603
2/2 - 1s - 305ms/step
[CV] END .............................l1_reg=0.01, n_units=5; total time=   3.7s
Epoch 1/20
4/4 - 10s - 3s/step - loss: 1.7774
Epoch 2/20
4/4 - 0s - 11ms/step - loss: 1.7470
Epoch 3/20
4/4 - 0s - 9ms/step - loss: 1.7253
Epoch 4/20
4/4 - 0s - 10ms/step - loss: 1.7037
Epoch 5/20
4/4 - 0s - 11ms/step - loss: 1.6772
Epoch 6/20
4/4 - 0s - 29ms/step - loss: 1.6574
Epoch 7/20
4/4 - 0s - 11ms/step - loss: 1.6390
Epoch 8/20
4/4 - 0s - 12ms/step - loss: 1.6184
Epoch 9/20
4/4 - 0s - 14ms/step - loss: 1.6025
Epoch 10/20
4/4 - 0s - 13ms/step - loss: 1.5869
Epoch 11/20
4/4 - 0s - 13ms/step - loss: 1.5698
Epoch 12/20
4/4 - 0s - 16ms/step - loss: 1.5549
Epoch 13/20
4/4 - 0s - 15ms/step - loss: 1.5395
Epoch 14/20
4/4 - 0s - 14ms/step - loss: 1.5194
Epoch 15/20
4/4 - 0s - 14ms/step - loss: 1.5078
Epoch 16/20
4/4 - 0s - 12ms/step - loss: 1.4977
Epoch 17/20
4/4 - 0s - 12ms/step - loss: 1.4917
Epoch 18/20
4/4 - 0s - 25ms/step - loss: 1.4789
Epoch 19/20
4/4 - 0s - 12ms/step - loss: 1.4727
Epoch 20/20
4/4 - 0s - 14ms/step - loss: 1.4637
2/2 - 1s - 360ms/step
[CV] END .............................l1_reg=0.01, n_units=5; total time=  12.3s
Epoch 1/20
5/5 - 2s - 459ms/step - loss: 1.5163
Epoch 2/20
5/5 - 0s - 8ms/step - loss: 1.4816
Epoch 3/20
5/5 - 0s - 8ms/step - loss: 1.4446
Epoch 4/20
5/5 - 0s - 7ms/step - loss: 1.4127
Epoch 5/20
5/5 - 0s - 7ms/step - loss: 1.3814
Epoch 6/20
5/5 - 0s - 6ms/step - loss: 1.3552
Epoch 7/20
5/5 - 0s - 7ms/step - loss: 1.3214
Epoch 8/20
5/5 - 0s - 7ms/step - loss: 1.3093
Epoch 9/20
5/5 - 0s - 8ms/step - loss: 1.2820
Epoch 10/20
5/5 - 0s - 10ms/step - loss: 1.2674
Epoch 11/20
5/5 - 0s - 7ms/step - loss: 1.2545
Epoch 12/20
5/5 - 0s - 19ms/step - loss: 1.2461
Epoch 13/20
5/5 - 0s - 8ms/step - loss: 1.2333
Epoch 14/20
5/5 - 0s - 8ms/step - loss: 1.2288
Epoch 15/20
5/5 - 0s - 8ms/step - loss: 1.2227
Epoch 16/20
5/5 - 0s - 8ms/step - loss: 1.2159
Epoch 17/20
5/5 - 0s - 10ms/step - loss: 1.2143
Epoch 18/20
5/5 - 0s - 11ms/step - loss: 1.2107
Epoch 19/20
5/5 - 0s - 10ms/step - loss: 1.2128
Epoch 20/20
5/5 - 0s - 9ms/step - loss: 1.2088
2/2 - 1s - 360ms/step
[CV] END .............................l1_reg=0.01, n_units=5; total time=   3.9s
Epoch 1/20
7/7 - 2s - 291ms/step - loss: 1.4325
Epoch 2/20
7/7 - 0s - 4ms/step - loss: 1.3929
Epoch 3/20
7/7 - 0s - 5ms/step - loss: 1.3570
Epoch 4/20
7/7 - 0s - 4ms/step - loss: 1.3216
Epoch 5/20
7/7 - 0s - 5ms/step - loss: 1.2890
Epoch 6/20
7/7 - 0s - 5ms/step - loss: 1.2647
Epoch 7/20
7/7 - 0s - 5ms/step - loss: 1.2548
Epoch 8/20
7/7 - 0s - 5ms/step - loss: 1.2356
Epoch 9/20
7/7 - 0s - 6ms/step - loss: 1.2174
Epoch 10/20
7/7 - 0s - 6ms/step - loss: 1.1987
Epoch 11/20
7/7 - 0s - 6ms/step - loss: 1.1830
Epoch 12/20
7/7 - 0s - 7ms/step - loss: 1.1700
Epoch 13/20
7/7 - 0s - 6ms/step - loss: 1.1549
Epoch 14/20
7/7 - 0s - 6ms/step - loss: 1.1441
Epoch 15/20
7/7 - 0s - 6ms/step - loss: 1.1388
Epoch 16/20
7/7 - 0s - 8ms/step - loss: 1.1350
Epoch 17/20
7/7 - 0s - 7ms/step - loss: 1.1355
Epoch 18/20
7/7 - 0s - 7ms/step - loss: 1.1303
Epoch 19/20
7/7 - 0s - 7ms/step - loss: 1.1311
Epoch 20/20
7/7 - 0s - 8ms/step - loss: 1.1292
2/2 - 1s - 306ms/step
[CV] END .............................l1_reg=0.01, n_units=5; total time=   3.5s
Epoch 1/20
8/8 - 2s - 263ms/step - loss: 1.2745
Epoch 2/20
8/8 - 0s - 5ms/step - loss: 1.2186
Epoch 3/20
8/8 - 0s - 7ms/step - loss: 1.1762
Epoch 4/20
8/8 - 0s - 5ms/step - loss: 1.1334
Epoch 5/20
8/8 - 0s - 6ms/step - loss: 1.0971
Epoch 6/20
8/8 - 0s - 7ms/step - loss: 1.0699
Epoch 7/20
8/8 - 0s - 6ms/step - loss: 1.0422
Epoch 8/20
8/8 - 0s - 6ms/step - loss: 1.0249
Epoch 9/20
8/8 - 0s - 6ms/step - loss: 1.0137
Epoch 10/20
8/8 - 0s - 6ms/step - loss: 1.0099
Epoch 11/20
8/8 - 0s - 7ms/step - loss: 1.0051
Epoch 12/20
8/8 - 0s - 8ms/step - loss: 1.0057
Epoch 13/20
8/8 - 0s - 7ms/step - loss: 1.0045
Epoch 14/20
8/8 - 0s - 13ms/step - loss: 1.0036
Epoch 15/20
8/8 - 0s - 8ms/step - loss: 1.0037
Epoch 16/20
8/8 - 0s - 7ms/step - loss: 1.0029
Epoch 17/20
8/8 - 0s - 7ms/step - loss: 1.0009
Epoch 18/20
8/8 - 0s - 7ms/step - loss: 1.0008
Epoch 19/20
8/8 - 0s - 7ms/step - loss: 1.0011
Epoch 20/20
8/8 - 0s - 7ms/step - loss: 1.0005
2/2 - 1s - 349ms/step
[CV] END .............................l1_reg=0.01, n_units=5; total time=   4.0s
Epoch 1/20
2/2 - 2s - 1s/step - loss: 1.6028
Epoch 2/20
2/2 - 0s - 20ms/step - loss: 1.5823
Epoch 3/20
2/2 - 0s - 15ms/step - loss: 1.5663
Epoch 4/20
2/2 - 0s - 15ms/step - loss: 1.5549
Epoch 5/20
2/2 - 0s - 14ms/step - loss: 1.5334
Epoch 6/20
2/2 - 0s - 14ms/step - loss: 1.5217
Epoch 7/20
2/2 - 0s - 14ms/step - loss: 1.5056
Epoch 8/20
2/2 - 0s - 14ms/step - loss: 1.4945
Epoch 9/20
2/2 - 0s - 13ms/step - loss: 1.4790
Epoch 10/20
2/2 - 0s - 19ms/step - loss: 1.4671
Epoch 11/20
2/2 - 0s - 19ms/step - loss: 1.4552
Epoch 12/20
2/2 - 0s - 15ms/step - loss: 1.4417
Epoch 13/20
2/2 - 0s - 19ms/step - loss: 1.4323
Epoch 14/20
2/2 - 0s - 20ms/step - loss: 1.4211
Epoch 15/20
2/2 - 0s - 18ms/step - loss: 1.4119
Epoch 16/20
2/2 - 0s - 15ms/step - loss: 1.4027
Epoch 17/20
2/2 - 0s - 19ms/step - loss: 1.4018
Epoch 18/20
2/2 - 0s - 13ms/step - loss: 1.3888
Epoch 19/20
2/2 - 0s - 17ms/step - loss: 1.3880
Epoch 20/20
2/2 - 0s - 20ms/step - loss: 1.3800
2/2 - 1s - 260ms/step
[CV] END ............................l1_reg=0.01, n_units=10; total time=   3.3s
Epoch 1/20
4/4 - 2s - 507ms/step - loss: 1.6644
Epoch 2/20
4/4 - 0s - 9ms/step - loss: 1.6356
Epoch 3/20
4/4 - 0s - 7ms/step - loss: 1.6047
Epoch 4/20
4/4 - 0s - 7ms/step - loss: 1.5839
Epoch 5/20
4/4 - 0s - 7ms/step - loss: 1.5673
Epoch 6/20
4/4 - 0s - 7ms/step - loss: 1.5455
Epoch 7/20
4/4 - 0s - 8ms/step - loss: 1.5382
Epoch 8/20
4/4 - 0s - 8ms/step - loss: 1.5253
Epoch 9/20
4/4 - 0s - 11ms/step - loss: 1.5206
Epoch 10/20
4/4 - 0s - 10ms/step - loss: 1.5099
Epoch 11/20
4/4 - 0s - 11ms/step - loss: 1.5049
Epoch 12/20
4/4 - 0s - 10ms/step - loss: 1.5063
Epoch 13/20
4/4 - 0s - 9ms/step - loss: 1.4997
Epoch 14/20
4/4 - 0s - 10ms/step - loss: 1.4996
Epoch 15/20
4/4 - 0s - 12ms/step - loss: 1.4985
Epoch 16/20
4/4 - 0s - 12ms/step - loss: 1.4975
Epoch 17/20
4/4 - 0s - 13ms/step - loss: 1.4958
Epoch 18/20
4/4 - 0s - 9ms/step - loss: 1.4914
Epoch 19/20
4/4 - 0s - 8ms/step - loss: 1.4920
Epoch 20/20
4/4 - 0s - 9ms/step - loss: 1.4916
2/2 - 1s - 289ms/step
[CV] END ............................l1_reg=0.01, n_units=10; total time=   3.4s
Epoch 1/20
5/5 - 2s - 403ms/step - loss: 1.4378
Epoch 2/20
5/5 - 0s - 6ms/step - loss: 1.3989
Epoch 3/20
5/5 - 0s - 8ms/step - loss: 1.3686
Epoch 4/20
5/5 - 0s - 5ms/step - loss: 1.3499
Epoch 5/20
5/5 - 0s - 6ms/step - loss: 1.3270
Epoch 6/20
5/5 - 0s - 6ms/step - loss: 1.3078
Epoch 7/20
5/5 - 0s - 8ms/step - loss: 1.2945
Epoch 8/20
5/5 - 0s - 6ms/step - loss: 1.2789
Epoch 9/20
5/5 - 0s - 9ms/step - loss: 1.2704
Epoch 10/20
5/5 - 0s - 10ms/step - loss: 1.2650
Epoch 11/20
5/5 - 0s - 11ms/step - loss: 1.2592
Epoch 12/20
5/5 - 0s - 8ms/step - loss: 1.2599
Epoch 13/20
5/5 - 0s - 11ms/step - loss: 1.2587
Epoch 14/20
5/5 - 0s - 8ms/step - loss: 1.2549
Epoch 15/20
5/5 - 0s - 8ms/step - loss: 1.2530
Epoch 16/20
5/5 - 0s - 22ms/step - loss: 1.2515
Epoch 17/20
5/5 - 0s - 8ms/step - loss: 1.2508
Epoch 18/20
5/5 - 0s - 8ms/step - loss: 1.2487
Epoch 19/20
5/5 - 0s - 7ms/step - loss: 1.2484
Epoch 20/20
5/5 - 0s - 8ms/step - loss: 1.2466
2/2 - 1s - 290ms/step
[CV] END ............................l1_reg=0.01, n_units=10; total time=   3.5s
Epoch 1/20
7/7 - 2s - 311ms/step - loss: 1.3400
Epoch 2/20
7/7 - 0s - 5ms/step - loss: 1.2869
Epoch 3/20
7/7 - 0s - 5ms/step - loss: 1.2456
Epoch 4/20
7/7 - 0s - 4ms/step - loss: 1.2263
Epoch 5/20
7/7 - 0s - 5ms/step - loss: 1.1953
Epoch 6/20
7/7 - 0s - 5ms/step - loss: 1.1938
Epoch 7/20
7/7 - 0s - 5ms/step - loss: 1.1921
Epoch 8/20
7/7 - 0s - 5ms/step - loss: 1.1891
Epoch 9/20
7/7 - 0s - 8ms/step - loss: 1.1828
Epoch 10/20
7/7 - 0s - 6ms/step - loss: 1.1781
Epoch 11/20
7/7 - 0s - 6ms/step - loss: 1.1803
Epoch 12/20
7/7 - 0s - 9ms/step - loss: 1.1759
Epoch 13/20
7/7 - 0s - 7ms/step - loss: 1.1768
Epoch 14/20
7/7 - 0s - 7ms/step - loss: 1.1754
Epoch 15/20
7/7 - 0s - 6ms/step - loss: 1.1743
Epoch 16/20
7/7 - 0s - 6ms/step - loss: 1.1749
Epoch 17/20
7/7 - 0s - 7ms/step - loss: 1.1722
Epoch 18/20
7/7 - 0s - 8ms/step - loss: 1.1684
Epoch 19/20
7/7 - 0s - 6ms/step - loss: 1.1679
Epoch 20/20
7/7 - 0s - 9ms/step - loss: 1.1674
2/2 - 1s - 284ms/step
[CV] END ............................l1_reg=0.01, n_units=10; total time=   3.7s
Epoch 1/20
8/8 - 2s - 263ms/step - loss: 1.2053
Epoch 2/20
8/8 - 0s - 4ms/step - loss: 1.1524
Epoch 3/20
8/8 - 0s - 5ms/step - loss: 1.1124
Epoch 4/20
8/8 - 0s - 5ms/step - loss: 1.0788
Epoch 5/20
8/8 - 0s - 5ms/step - loss: 1.0636
Epoch 6/20
8/8 - 0s - 6ms/step - loss: 1.0565
Epoch 7/20
8/8 - 0s - 6ms/step - loss: 1.0504
Epoch 8/20
8/8 - 0s - 7ms/step - loss: 1.0504
Epoch 9/20
8/8 - 0s - 5ms/step - loss: 1.0472
Epoch 10/20
8/8 - 0s - 5ms/step - loss: 1.0456
Epoch 11/20
8/8 - 0s - 6ms/step - loss: 1.0445
Epoch 12/20
8/8 - 0s - 6ms/step - loss: 1.0422
Epoch 13/20
8/8 - 0s - 7ms/step - loss: 1.0403
Epoch 14/20
8/8 - 0s - 6ms/step - loss: 1.0390
Epoch 15/20
8/8 - 0s - 7ms/step - loss: 1.0370
Epoch 16/20
8/8 - 0s - 8ms/step - loss: 1.0384
Epoch 17/20
8/8 - 0s - 8ms/step - loss: 1.0353
Epoch 18/20
8/8 - 0s - 8ms/step - loss: 1.0332
Epoch 19/20
8/8 - 0s - 7ms/step - loss: 1.0323
Epoch 20/20
8/8 - 0s - 7ms/step - loss: 1.0316
2/2 - 1s - 262ms/step
[CV] END ............................l1_reg=0.01, n_units=10; total time=   3.6s
Epoch 1/20
2/2 - 2s - 1s/step - loss: 1.7123
Epoch 2/20
2/2 - 0s - 16ms/step - loss: 1.6933
Epoch 3/20
2/2 - 0s - 13ms/step - loss: 1.6700
Epoch 4/20
2/2 - 0s - 15ms/step - loss: 1.6442
Epoch 5/20
2/2 - 0s - 14ms/step - loss: 1.6259
Epoch 6/20
2/2 - 0s - 13ms/step - loss: 1.6075
Epoch 7/20
2/2 - 0s - 12ms/step - loss: 1.5830
Epoch 8/20
2/2 - 0s - 12ms/step - loss: 1.5720
Epoch 9/20
2/2 - 0s - 14ms/step - loss: 1.5483
Epoch 10/20
2/2 - 0s - 14ms/step - loss: 1.5330
Epoch 11/20
2/2 - 0s - 16ms/step - loss: 1.5201
Epoch 12/20
2/2 - 0s - 16ms/step - loss: 1.5050
Epoch 13/20
2/2 - 0s - 17ms/step - loss: 1.4929
Epoch 14/20
2/2 - 0s - 13ms/step - loss: 1.4894
Epoch 15/20
2/2 - 0s - 13ms/step - loss: 1.4810
Epoch 16/20
2/2 - 0s - 16ms/step - loss: 1.4739
Epoch 17/20
2/2 - 0s - 21ms/step - loss: 1.4622
Epoch 18/20
2/2 - 0s - 14ms/step - loss: 1.4570
Epoch 19/20
2/2 - 0s - 23ms/step - loss: 1.4521
Epoch 20/20
2/2 - 0s - 14ms/step - loss: 1.4419
2/2 - 0s - 225ms/step
[CV] END ............................l1_reg=0.01, n_units=20; total time=   3.2s
Epoch 1/20
4/4 - 2s - 487ms/step - loss: 1.7195
Epoch 2/20
4/4 - 0s - 7ms/step - loss: 1.6834
Epoch 3/20
4/4 - 0s - 8ms/step - loss: 1.6615
Epoch 4/20
4/4 - 0s - 10ms/step - loss: 1.6467
Epoch 5/20
4/4 - 0s - 8ms/step - loss: 1.6171
Epoch 6/20
4/4 - 0s - 8ms/step - loss: 1.6062
Epoch 7/20
4/4 - 0s - 9ms/step - loss: 1.5968
Epoch 8/20
4/4 - 0s - 8ms/step - loss: 1.5932
Epoch 9/20
4/4 - 0s - 8ms/step - loss: 1.5916
Epoch 10/20
4/4 - 0s - 8ms/step - loss: 1.5850
Epoch 11/20
4/4 - 0s - 9ms/step - loss: 1.5932
Epoch 12/20
4/4 - 0s - 9ms/step - loss: 1.5841
Epoch 13/20
4/4 - 0s - 12ms/step - loss: 1.5827
Epoch 14/20
4/4 - 0s - 8ms/step - loss: 1.5858
Epoch 15/20
4/4 - 0s - 10ms/step - loss: 1.5819
Epoch 16/20
4/4 - 0s - 8ms/step - loss: 1.5761
Epoch 17/20
4/4 - 0s - 8ms/step - loss: 1.5768
Epoch 18/20
4/4 - 0s - 12ms/step - loss: 1.5756
Epoch 19/20
4/4 - 0s - 12ms/step - loss: 1.5756
Epoch 20/20
4/4 - 0s - 12ms/step - loss: 1.5737
2/2 - 1s - 262ms/step
[CV] END ............................l1_reg=0.01, n_units=20; total time=   3.2s
Epoch 1/20
5/5 - 2s - 421ms/step - loss: 1.4246
Epoch 2/20
5/5 - 0s - 6ms/step - loss: 1.3798
Epoch 3/20
5/5 - 0s - 8ms/step - loss: 1.3588
Epoch 4/20
5/5 - 0s - 7ms/step - loss: 1.3414
Epoch 5/20
5/5 - 0s - 7ms/step - loss: 1.3323
Epoch 6/20
5/5 - 0s - 7ms/step - loss: 1.3129
Epoch 7/20
5/5 - 0s - 6ms/step - loss: 1.3110
Epoch 8/20
5/5 - 0s - 7ms/step - loss: 1.3110
Epoch 9/20
5/5 - 0s - 7ms/step - loss: 1.3058
Epoch 10/20
5/5 - 0s - 7ms/step - loss: 1.3025
Epoch 11/20
5/5 - 0s - 7ms/step - loss: 1.3004
Epoch 12/20
5/5 - 0s - 10ms/step - loss: 1.2975
Epoch 13/20
5/5 - 0s - 9ms/step - loss: 1.2962
Epoch 14/20
5/5 - 0s - 9ms/step - loss: 1.2967
Epoch 15/20
5/5 - 0s - 10ms/step - loss: 1.2923
Epoch 16/20
5/5 - 0s - 8ms/step - loss: 1.2892
Epoch 17/20
5/5 - 0s - 8ms/step - loss: 1.2876
Epoch 18/20
5/5 - 0s - 8ms/step - loss: 1.2867
Epoch 19/20
5/5 - 0s - 10ms/step - loss: 1.2889
Epoch 20/20
5/5 - 0s - 11ms/step - loss: 1.2842
2/2 - 1s - 327ms/step
[CV] END ............................l1_reg=0.01, n_units=20; total time=   3.6s
Epoch 1/20
7/7 - 3s - 469ms/step - loss: 1.3891
Epoch 2/20
7/7 - 0s - 6ms/step - loss: 1.3506
Epoch 3/20
7/7 - 0s - 6ms/step - loss: 1.3083
Epoch 4/20
7/7 - 0s - 6ms/step - loss: 1.2838
Epoch 5/20
7/7 - 0s - 6ms/step - loss: 1.2608
Epoch 6/20
7/7 - 0s - 6ms/step - loss: 1.2491
Epoch 7/20
7/7 - 0s - 7ms/step - loss: 1.2444
Epoch 8/20
7/7 - 0s - 7ms/step - loss: 1.2408
Epoch 9/20
7/7 - 0s - 8ms/step - loss: 1.2438
Epoch 10/20
7/7 - 0s - 8ms/step - loss: 1.2364
Epoch 11/20
7/7 - 0s - 8ms/step - loss: 1.2307
Epoch 12/20
7/7 - 0s - 8ms/step - loss: 1.2315
Epoch 13/20
7/7 - 0s - 7ms/step - loss: 1.2374
Epoch 14/20
7/7 - 0s - 8ms/step - loss: 1.2301
Epoch 15/20
7/7 - 0s - 8ms/step - loss: 1.2223
Epoch 16/20
7/7 - 0s - 8ms/step - loss: 1.2210
Epoch 17/20
7/7 - 0s - 7ms/step - loss: 1.2100
Epoch 18/20
7/7 - 0s - 8ms/step - loss: 1.2093
Epoch 19/20
7/7 - 0s - 8ms/step - loss: 1.2055
Epoch 20/20
7/7 - 0s - 9ms/step - loss: 1.2141
2/2 - 1s - 326ms/step
[CV] END ............................l1_reg=0.01, n_units=20; total time=   5.0s
Epoch 1/20
8/8 - 2s - 277ms/step - loss: 1.2117
Epoch 2/20
8/8 - 0s - 5ms/step - loss: 1.1531
Epoch 3/20
8/8 - 0s - 5ms/step - loss: 1.1277
Epoch 4/20
8/8 - 0s - 5ms/step - loss: 1.1224
Epoch 5/20
8/8 - 0s - 5ms/step - loss: 1.1159
Epoch 6/20
8/8 - 0s - 5ms/step - loss: 1.1140
Epoch 7/20
8/8 - 0s - 5ms/step - loss: 1.1058
Epoch 8/20
8/8 - 0s - 6ms/step - loss: 1.1008
Epoch 9/20
8/8 - 0s - 6ms/step - loss: 1.0966
Epoch 10/20
8/8 - 0s - 5ms/step - loss: 1.0925
Epoch 11/20
8/8 - 0s - 5ms/step - loss: 1.0921
Epoch 12/20
8/8 - 0s - 8ms/step - loss: 1.0855
Epoch 13/20
8/8 - 0s - 6ms/step - loss: 1.0827
Epoch 14/20
8/8 - 0s - 6ms/step - loss: 1.0772
Epoch 15/20
8/8 - 0s - 5ms/step - loss: 1.0762
Epoch 16/20
8/8 - 0s - 6ms/step - loss: 1.0699
Epoch 17/20
8/8 - 0s - 6ms/step - loss: 1.0684
Epoch 18/20
8/8 - 0s - 7ms/step - loss: 1.0644
Epoch 19/20
8/8 - 0s - 6ms/step - loss: 1.0672
Epoch 20/20
8/8 - 0s - 5ms/step - loss: 1.0615
2/2 - 1s - 318ms/step
[CV] END ............................l1_reg=0.01, n_units=20; total time=   3.8s
Epoch 1/20
2/2 - 3s - 1s/step - loss: 2.4655
Epoch 2/20
2/2 - 0s - 24ms/step - loss: 2.4488
Epoch 3/20
2/2 - 0s - 24ms/step - loss: 2.4308
Epoch 4/20
2/2 - 0s - 28ms/step - loss: 2.4139
Epoch 5/20
2/2 - 0s - 25ms/step - loss: 2.3971
Epoch 6/20
2/2 - 0s - 25ms/step - loss: 2.3785
Epoch 7/20
2/2 - 0s - 36ms/step - loss: 2.3620
Epoch 8/20
2/2 - 0s - 26ms/step - loss: 2.3476
Epoch 9/20
2/2 - 0s - 28ms/step - loss: 2.3265
Epoch 10/20
2/2 - 0s - 26ms/step - loss: 2.3163
Epoch 11/20
2/2 - 0s - 30ms/step - loss: 2.3001
Epoch 12/20
2/2 - 0s - 27ms/step - loss: 2.2836
Epoch 13/20
2/2 - 0s - 33ms/step - loss: 2.2652
Epoch 14/20
2/2 - 0s - 30ms/step - loss: 2.2550
Epoch 15/20
2/2 - 0s - 31ms/step - loss: 2.2444
Epoch 16/20
2/2 - 0s - 30ms/step - loss: 2.2318
Epoch 17/20
2/2 - 0s - 29ms/step - loss: 2.2199
Epoch 18/20
2/2 - 0s - 30ms/step - loss: 2.2064
Epoch 19/20
2/2 - 0s - 29ms/step - loss: 2.1935
Epoch 20/20
2/2 - 0s - 27ms/step - loss: 2.1832
2/2 - 1s - 482ms/step
[CV] END ..............................l1_reg=0.1, n_units=5; total time=   4.9s
Epoch 1/20
4/4 - 3s - 680ms/step - loss: 2.4706
Epoch 2/20
4/4 - 0s - 10ms/step - loss: 2.4372
Epoch 3/20
4/4 - 0s - 11ms/step - loss: 2.4003
Epoch 4/20
4/4 - 0s - 25ms/step - loss: 2.3709
Epoch 5/20
4/4 - 0s - 8ms/step - loss: 2.3481
Epoch 6/20
4/4 - 0s - 8ms/step - loss: 2.3243
Epoch 7/20
4/4 - 0s - 8ms/step - loss: 2.3016
Epoch 8/20
4/4 - 0s - 8ms/step - loss: 2.2828
Epoch 9/20
4/4 - 0s - 9ms/step - loss: 2.2647
Epoch 10/20
4/4 - 0s - 9ms/step - loss: 2.2468
Epoch 11/20
4/4 - 0s - 10ms/step - loss: 2.2342
Epoch 12/20
4/4 - 0s - 9ms/step - loss: 2.2207
Epoch 13/20
4/4 - 0s - 10ms/step - loss: 2.2092
Epoch 14/20
4/4 - 0s - 13ms/step - loss: 2.1960
Epoch 15/20
4/4 - 0s - 14ms/step - loss: 2.1799
Epoch 16/20
4/4 - 0s - 11ms/step - loss: 2.1683
Epoch 17/20
4/4 - 0s - 20ms/step - loss: 2.1602
Epoch 18/20
4/4 - 0s - 12ms/step - loss: 2.1505
Epoch 19/20
4/4 - 0s - 12ms/step - loss: 2.1423
Epoch 20/20
4/4 - 0s - 11ms/step - loss: 2.1321
2/2 - 1s - 310ms/step
[CV] END ..............................l1_reg=0.1, n_units=5; total time=   4.3s
Epoch 1/20
5/5 - 3s - 520ms/step - loss: 2.2053
Epoch 2/20
5/5 - 0s - 11ms/step - loss: 2.1725
Epoch 3/20
5/5 - 0s - 11ms/step - loss: 2.1374
Epoch 4/20
5/5 - 0s - 12ms/step - loss: 2.0984
Epoch 5/20
5/5 - 0s - 11ms/step - loss: 2.0637
Epoch 6/20
5/5 - 0s - 12ms/step - loss: 2.0297
Epoch 7/20
5/5 - 0s - 15ms/step - loss: 1.9981
Epoch 8/20
5/5 - 0s - 11ms/step - loss: 1.9728
Epoch 9/20
5/5 - 0s - 10ms/step - loss: 1.9421
Epoch 10/20
5/5 - 0s - 11ms/step - loss: 1.9113
Epoch 11/20
5/5 - 0s - 10ms/step - loss: 1.8959
Epoch 12/20
5/5 - 0s - 10ms/step - loss: 1.8677
Epoch 13/20
5/5 - 0s - 11ms/step - loss: 1.8508
Epoch 14/20
5/5 - 0s - 11ms/step - loss: 1.8382
Epoch 15/20
5/5 - 0s - 9ms/step - loss: 1.8177
Epoch 16/20
5/5 - 0s - 9ms/step - loss: 1.8037
Epoch 17/20
5/5 - 0s - 11ms/step - loss: 1.7913
Epoch 18/20
5/5 - 0s - 10ms/step - loss: 1.7796
Epoch 19/20
5/5 - 0s - 10ms/step - loss: 1.7716
Epoch 20/20
5/5 - 0s - 10ms/step - loss: 1.7615
2/2 - 1s - 356ms/step
[CV] END ..............................l1_reg=0.1, n_units=5; total time=   4.4s
Epoch 1/20
7/7 - 2s - 309ms/step - loss: 2.1180
Epoch 2/20
7/7 - 0s - 5ms/step - loss: 2.0632
Epoch 3/20
7/7 - 0s - 5ms/step - loss: 2.0181
Epoch 4/20
7/7 - 0s - 4ms/step - loss: 1.9677
Epoch 5/20
7/7 - 0s - 6ms/step - loss: 1.9197
Epoch 6/20
7/7 - 0s - 5ms/step - loss: 1.8893
Epoch 7/20
7/7 - 0s - 5ms/step - loss: 1.8595
Epoch 8/20
7/7 - 0s - 6ms/step - loss: 1.8353
Epoch 9/20
7/7 - 0s - 7ms/step - loss: 1.8126
Epoch 10/20
7/7 - 0s - 5ms/step - loss: 1.7896
Epoch 11/20
7/7 - 0s - 7ms/step - loss: 1.7727
Epoch 12/20
7/7 - 0s - 6ms/step - loss: 1.7576
Epoch 13/20
7/7 - 0s - 6ms/step - loss: 1.7396
Epoch 14/20
7/7 - 0s - 6ms/step - loss: 1.7285
Epoch 15/20
7/7 - 0s - 8ms/step - loss: 1.7130
Epoch 16/20
7/7 - 0s - 5ms/step - loss: 1.6987
Epoch 17/20
7/7 - 0s - 5ms/step - loss: 1.6856
Epoch 18/20
7/7 - 0s - 5ms/step - loss: 1.6740
Epoch 19/20
7/7 - 0s - 5ms/step - loss: 1.6591
Epoch 20/20
7/7 - 0s - 7ms/step - loss: 1.6511
2/2 - 1s - 266ms/step
[CV] END ..............................l1_reg=0.1, n_units=5; total time=   3.5s
Epoch 1/20
8/8 - 2s - 262ms/step - loss: 1.9617
Epoch 2/20
8/8 - 0s - 4ms/step - loss: 1.8992
Epoch 3/20
8/8 - 0s - 4ms/step - loss: 1.8484
Epoch 4/20
8/8 - 0s - 5ms/step - loss: 1.7944
Epoch 5/20
8/8 - 0s - 5ms/step - loss: 1.7482
Epoch 6/20
8/8 - 0s - 4ms/step - loss: 1.7060
Epoch 7/20
8/8 - 0s - 5ms/step - loss: 1.6744
Epoch 8/20
8/8 - 0s - 5ms/step - loss: 1.6441
Epoch 9/20
8/8 - 0s - 5ms/step - loss: 1.6199
Epoch 10/20
8/8 - 0s - 6ms/step - loss: 1.6004
Epoch 11/20
8/8 - 0s - 7ms/step - loss: 1.5823
Epoch 12/20
8/8 - 0s - 5ms/step - loss: 1.5698
Epoch 13/20
8/8 - 0s - 7ms/step - loss: 1.5557
Epoch 14/20
8/8 - 0s - 6ms/step - loss: 1.5436
Epoch 15/20
8/8 - 0s - 6ms/step - loss: 1.5306
Epoch 16/20
8/8 - 0s - 6ms/step - loss: 1.5202
Epoch 17/20
8/8 - 0s - 6ms/step - loss: 1.5103
Epoch 18/20
8/8 - 0s - 6ms/step - loss: 1.4970
Epoch 19/20
8/8 - 0s - 7ms/step - loss: 1.4856
Epoch 20/20
8/8 - 0s - 6ms/step - loss: 1.4763
2/2 - 1s - 375ms/step
[CV] END ..............................l1_reg=0.1, n_units=5; total time=   3.7s
Epoch 1/20
2/2 - 2s - 1s/step - loss: 2.6516
Epoch 2/20
2/2 - 0s - 12ms/step - loss: 2.6256
Epoch 3/20
2/2 - 0s - 15ms/step - loss: 2.6094
Epoch 4/20
2/2 - 0s - 13ms/step - loss: 2.5893
Epoch 5/20
2/2 - 0s - 13ms/step - loss: 2.5661
Epoch 6/20
2/2 - 0s - 18ms/step - loss: 2.5460
Epoch 7/20
2/2 - 0s - 13ms/step - loss: 2.5274
Epoch 8/20
2/2 - 0s - 14ms/step - loss: 2.5088
Epoch 9/20
2/2 - 0s - 14ms/step - loss: 2.4887
Epoch 10/20
2/2 - 0s - 15ms/step - loss: 2.4740
Epoch 11/20
2/2 - 0s - 18ms/step - loss: 2.4563
Epoch 12/20
2/2 - 0s - 18ms/step - loss: 2.4410
Epoch 13/20
2/2 - 0s - 19ms/step - loss: 2.4259
Epoch 14/20
2/2 - 0s - 15ms/step - loss: 2.4094
Epoch 15/20
2/2 - 0s - 20ms/step - loss: 2.3935
Epoch 16/20
2/2 - 0s - 20ms/step - loss: 2.3772
Epoch 17/20
2/2 - 0s - 14ms/step - loss: 2.3650
Epoch 18/20
2/2 - 0s - 15ms/step - loss: 2.3528
Epoch 19/20
2/2 - 0s - 16ms/step - loss: 2.3412
Epoch 20/20
2/2 - 0s - 26ms/step - loss: 2.3344
2/2 - 1s - 369ms/step
[CV] END .............................l1_reg=0.1, n_units=10; total time=   3.8s
Epoch 1/20
4/4 - 2s - 619ms/step - loss: 2.7181
Epoch 2/20
4/4 - 0s - 9ms/step - loss: 2.6829
Epoch 3/20
4/4 - 0s - 9ms/step - loss: 2.6579
Epoch 4/20
4/4 - 0s - 10ms/step - loss: 2.6347
Epoch 5/20
4/4 - 0s - 12ms/step - loss: 2.6065
Epoch 6/20
4/4 - 0s - 11ms/step - loss: 2.5759
Epoch 7/20
4/4 - 0s - 11ms/step - loss: 2.5494
Epoch 8/20
4/4 - 0s - 10ms/step - loss: 2.5294
Epoch 9/20
4/4 - 0s - 11ms/step - loss: 2.5042
Epoch 10/20
4/4 - 0s - 11ms/step - loss: 2.4787
Epoch 11/20
4/4 - 0s - 12ms/step - loss: 2.4589
Epoch 12/20
4/4 - 0s - 16ms/step - loss: 2.4313
Epoch 13/20
4/4 - 0s - 21ms/step - loss: 2.4170
Epoch 14/20
4/4 - 0s - 13ms/step - loss: 2.3901
Epoch 15/20
4/4 - 0s - 15ms/step - loss: 2.3700
Epoch 16/20
4/4 - 0s - 14ms/step - loss: 2.3556
Epoch 17/20
4/4 - 0s - 16ms/step - loss: 2.3354
Epoch 18/20
4/4 - 0s - 17ms/step - loss: 2.3270
Epoch 19/20
4/4 - 0s - 15ms/step - loss: 2.3159
Epoch 20/20
4/4 - 0s - 15ms/step - loss: 2.3064
2/2 - 1s - 410ms/step
[CV] END .............................l1_reg=0.1, n_units=10; total time=   4.4s
Epoch 1/20
5/5 - 3s - 508ms/step - loss: 2.4859
Epoch 2/20
5/5 - 0s - 8ms/step - loss: 2.4289
Epoch 3/20
5/5 - 0s - 9ms/step - loss: 2.3776
Epoch 4/20
5/5 - 0s - 8ms/step - loss: 2.3345
Epoch 5/20
5/5 - 0s - 9ms/step - loss: 2.2955
Epoch 6/20
5/5 - 0s - 10ms/step - loss: 2.2604
Epoch 7/20
5/5 - 0s - 10ms/step - loss: 2.2257
Epoch 8/20
5/5 - 0s - 10ms/step - loss: 2.1992
Epoch 9/20
5/5 - 0s - 10ms/step - loss: 2.1686
Epoch 10/20
5/5 - 0s - 10ms/step - loss: 2.1447
Epoch 11/20
5/5 - 0s - 10ms/step - loss: 2.1265
Epoch 12/20
5/5 - 0s - 10ms/step - loss: 2.1025
Epoch 13/20
5/5 - 0s - 9ms/step - loss: 2.0835
Epoch 14/20
5/5 - 0s - 10ms/step - loss: 2.0633
Epoch 15/20
5/5 - 0s - 11ms/step - loss: 2.0445
Epoch 16/20
5/5 - 0s - 10ms/step - loss: 2.0260
Epoch 17/20
5/5 - 0s - 10ms/step - loss: 2.0099
Epoch 18/20
5/5 - 0s - 10ms/step - loss: 1.9928
Epoch 19/20
5/5 - 0s - 10ms/step - loss: 1.9732
Epoch 20/20
5/5 - 0s - 10ms/step - loss: 1.9562
2/2 - 1s - 377ms/step
[CV] END .............................l1_reg=0.1, n_units=10; total time=   4.3s
Epoch 1/20
7/7 - 3s - 429ms/step - loss: 2.3946
Epoch 2/20
7/7 - 0s - 7ms/step - loss: 2.3313
Epoch 3/20
7/7 - 0s - 7ms/step - loss: 2.2604
Epoch 4/20
7/7 - 0s - 6ms/step - loss: 2.2086
Epoch 5/20
7/7 - 0s - 6ms/step - loss: 2.1620
Epoch 6/20
7/7 - 0s - 7ms/step - loss: 2.1175
Epoch 7/20
7/7 - 0s - 6ms/step - loss: 2.0851
Epoch 8/20
7/7 - 0s - 6ms/step - loss: 2.0629
Epoch 9/20
7/7 - 0s - 7ms/step - loss: 2.0378
Epoch 10/20
7/7 - 0s - 7ms/step - loss: 2.0149
Epoch 11/20
7/7 - 0s - 8ms/step - loss: 1.9928
Epoch 12/20
7/7 - 0s - 10ms/step - loss: 1.9765
Epoch 13/20
7/7 - 0s - 8ms/step - loss: 1.9560
Epoch 14/20
7/7 - 0s - 8ms/step - loss: 1.9349
Epoch 15/20
7/7 - 0s - 13ms/step - loss: 1.9139
Epoch 16/20
7/7 - 0s - 10ms/step - loss: 1.8921
Epoch 17/20
7/7 - 0s - 9ms/step - loss: 1.8739
Epoch 18/20
7/7 - 0s - 10ms/step - loss: 1.8531
Epoch 19/20
7/7 - 0s - 11ms/step - loss: 1.8341
Epoch 20/20
7/7 - 0s - 9ms/step - loss: 1.8152
2/2 - 1s - 431ms/step
[CV] END .............................l1_reg=0.1, n_units=10; total time=   5.1s
Epoch 1/20
8/8 - 3s - 351ms/step - loss: 2.2405
Epoch 2/20
8/8 - 0s - 6ms/step - loss: 2.1562
Epoch 3/20
8/8 - 0s - 7ms/step - loss: 2.0961
Epoch 4/20
8/8 - 0s - 6ms/step - loss: 2.0316
Epoch 5/20
8/8 - 0s - 7ms/step - loss: 1.9859
Epoch 6/20
8/8 - 0s - 6ms/step - loss: 1.9469
Epoch 7/20
8/8 - 0s - 7ms/step - loss: 1.9126
Epoch 8/20
8/8 - 0s - 8ms/step - loss: 1.8813
Epoch 9/20
8/8 - 0s - 8ms/step - loss: 1.8489
Epoch 10/20
8/8 - 0s - 7ms/step - loss: 1.8194
Epoch 11/20
8/8 - 0s - 8ms/step - loss: 1.7904
Epoch 12/20
8/8 - 0s - 8ms/step - loss: 1.7644
Epoch 13/20
8/8 - 0s - 7ms/step - loss: 1.7362
Epoch 14/20
8/8 - 0s - 14ms/step - loss: 1.7106
Epoch 15/20
8/8 - 0s - 8ms/step - loss: 1.6845
Epoch 16/20
8/8 - 0s - 9ms/step - loss: 1.6597
Epoch 17/20
8/8 - 0s - 9ms/step - loss: 1.6354
Epoch 18/20
8/8 - 0s - 8ms/step - loss: 1.6072
Epoch 19/20
8/8 - 0s - 8ms/step - loss: 1.5851
Epoch 20/20
8/8 - 0s - 8ms/step - loss: 1.5599
2/2 - 1s - 395ms/step
[CV] END .............................l1_reg=0.1, n_units=10; total time=   4.9s
Epoch 1/20
2/2 - 3s - 2s/step - loss: 3.1940
Epoch 2/20
2/2 - 0s - 17ms/step - loss: 3.1554
Epoch 3/20
2/2 - 0s - 16ms/step - loss: 3.1280
Epoch 4/20
2/2 - 0s - 17ms/step - loss: 3.0999
Epoch 5/20
2/2 - 0s - 18ms/step - loss: 3.0654
Epoch 6/20
2/2 - 0s - 20ms/step - loss: 3.0344
Epoch 7/20
2/2 - 0s - 22ms/step - loss: 3.0022
Epoch 8/20
2/2 - 0s - 16ms/step - loss: 2.9799
Epoch 9/20
2/2 - 0s - 18ms/step - loss: 2.9419
Epoch 10/20
2/2 - 0s - 19ms/step - loss: 2.9223
Epoch 11/20
2/2 - 0s - 16ms/step - loss: 2.8959
Epoch 12/20
2/2 - 0s - 19ms/step - loss: 2.8675
Epoch 13/20
2/2 - 0s - 22ms/step - loss: 2.8447
Epoch 14/20
2/2 - 0s - 17ms/step - loss: 2.8204
Epoch 15/20
2/2 - 0s - 18ms/step - loss: 2.7971
Epoch 16/20
2/2 - 0s - 18ms/step - loss: 2.7831
Epoch 17/20
2/2 - 0s - 59ms/step - loss: 2.7556
Epoch 18/20
2/2 - 0s - 24ms/step - loss: 2.7361
Epoch 19/20
2/2 - 0s - 27ms/step - loss: 2.7176
Epoch 20/20
2/2 - 0s - 28ms/step - loss: 2.7053
2/2 - 1s - 441ms/step
[CV] END .............................l1_reg=0.1, n_units=20; total time=   4.9s
Epoch 1/20
4/4 - 3s - 672ms/step - loss: 3.1948
Epoch 2/20
4/4 - 0s - 8ms/step - loss: 3.1358
Epoch 3/20
4/4 - 0s - 8ms/step - loss: 3.0909
Epoch 4/20
4/4 - 0s - 8ms/step - loss: 3.0573
Epoch 5/20
4/4 - 0s - 9ms/step - loss: 3.0192
Epoch 6/20
4/4 - 0s - 11ms/step - loss: 2.9744
Epoch 7/20
4/4 - 0s - 10ms/step - loss: 2.9350
Epoch 8/20
4/4 - 0s - 11ms/step - loss: 2.8943
Epoch 9/20
4/4 - 0s - 10ms/step - loss: 2.8543
Epoch 10/20
4/4 - 0s - 11ms/step - loss: 2.8186
Epoch 11/20
4/4 - 0s - 10ms/step - loss: 2.7887
Epoch 12/20
4/4 - 0s - 11ms/step - loss: 2.7601
Epoch 13/20
4/4 - 0s - 11ms/step - loss: 2.7225
Epoch 14/20
4/4 - 0s - 9ms/step - loss: 2.7072
Epoch 15/20
4/4 - 0s - 13ms/step - loss: 2.6871
Epoch 16/20
4/4 - 0s - 12ms/step - loss: 2.6772
Epoch 17/20
4/4 - 0s - 14ms/step - loss: 2.6758
Epoch 18/20
4/4 - 0s - 13ms/step - loss: 2.6618
Epoch 19/20
4/4 - 0s - 12ms/step - loss: 2.6323
Epoch 20/20
4/4 - 0s - 12ms/step - loss: 2.5988
2/2 - 8s - 4s/step
[CV] END .............................l1_reg=0.1, n_units=20; total time=  11.8s
Epoch 1/20
5/5 - 3s - 644ms/step - loss: 2.9020
Epoch 2/20
5/5 - 0s - 9ms/step - loss: 2.8261
Epoch 3/20
5/5 - 0s - 10ms/step - loss: 2.7648
Epoch 4/20
5/5 - 0s - 9ms/step - loss: 2.6968
Epoch 5/20
5/5 - 0s - 11ms/step - loss: 2.6484
Epoch 6/20
5/5 - 0s - 10ms/step - loss: 2.5935
Epoch 7/20
5/5 - 0s - 11ms/step - loss: 2.5475
Epoch 8/20
5/5 - 0s - 10ms/step - loss: 2.5054
Epoch 9/20
5/5 - 0s - 13ms/step - loss: 2.4636
Epoch 10/20
5/5 - 0s - 13ms/step - loss: 2.4236
Epoch 11/20
5/5 - 0s - 12ms/step - loss: 2.3841
Epoch 12/20
5/5 - 0s - 12ms/step - loss: 2.3460
Epoch 13/20
5/5 - 0s - 22ms/step - loss: 2.3083
Epoch 14/20
5/5 - 0s - 11ms/step - loss: 2.2737
Epoch 15/20
5/5 - 0s - 11ms/step - loss: 2.2388
Epoch 16/20
5/5 - 0s - 12ms/step - loss: 2.2025
Epoch 17/20
5/5 - 0s - 12ms/step - loss: 2.1730
Epoch 18/20
5/5 - 0s - 12ms/step - loss: 2.1346
Epoch 19/20
5/5 - 0s - 12ms/step - loss: 2.1020
Epoch 20/20
5/5 - 0s - 13ms/step - loss: 2.0715
2/2 - 1s - 423ms/step
[CV] END .............................l1_reg=0.1, n_units=20; total time=   5.3s
Epoch 1/20
7/7 - 2s - 346ms/step - loss: 2.8383
Epoch 2/20
7/7 - 0s - 5ms/step - loss: 2.7376
Epoch 3/20
7/7 - 0s - 6ms/step - loss: 2.6566
Epoch 4/20
7/7 - 0s - 5ms/step - loss: 2.5842
Epoch 5/20
7/7 - 0s - 6ms/step - loss: 2.5207
Epoch 6/20
7/7 - 0s - 6ms/step - loss: 2.4617
Epoch 7/20
7/7 - 0s - 6ms/step - loss: 2.4109
Epoch 8/20
7/7 - 0s - 6ms/step - loss: 2.3615
Epoch 9/20
7/7 - 0s - 8ms/step - loss: 2.3094
Epoch 10/20
7/7 - 0s - 8ms/step - loss: 2.2614
Epoch 11/20
7/7 - 0s - 8ms/step - loss: 2.2189
Epoch 12/20
7/7 - 0s - 7ms/step - loss: 2.1676
Epoch 13/20
7/7 - 0s - 5ms/step - loss: 2.1234
Epoch 14/20
7/7 - 0s - 6ms/step - loss: 2.0789
Epoch 15/20
7/7 - 0s - 8ms/step - loss: 2.0400
Epoch 16/20
7/7 - 0s - 7ms/step - loss: 1.9930
Epoch 17/20
7/7 - 0s - 7ms/step - loss: 1.9559
Epoch 18/20
7/7 - 0s - 5ms/step - loss: 1.9177
Epoch 19/20
7/7 - 0s - 7ms/step - loss: 1.8781
Epoch 20/20
7/7 - 0s - 7ms/step - loss: 1.8451
2/2 - 1s - 390ms/step
[CV] END .............................l1_reg=0.1, n_units=20; total time=   4.2s
Epoch 1/20
8/8 - 2s - 293ms/step - loss: 2.6911
Epoch 2/20
8/8 - 0s - 9ms/step - loss: 2.5653
Epoch 3/20
8/8 - 0s - 6ms/step - loss: 2.4794
Epoch 4/20
8/8 - 0s - 5ms/step - loss: 2.4037
Epoch 5/20
8/8 - 0s - 5ms/step - loss: 2.3260
Epoch 6/20
8/8 - 0s - 5ms/step - loss: 2.2625
Epoch 7/20
8/8 - 0s - 6ms/step - loss: 2.1963
Epoch 8/20
8/8 - 0s - 7ms/step - loss: 2.1351
Epoch 9/20
8/8 - 0s - 7ms/step - loss: 2.0770
Epoch 10/20
8/8 - 0s - 6ms/step - loss: 2.0187
Epoch 11/20
8/8 - 0s - 6ms/step - loss: 1.9639
Epoch 12/20
8/8 - 0s - 6ms/step - loss: 1.9093
Epoch 13/20
8/8 - 0s - 6ms/step - loss: 1.8541
Epoch 14/20
8/8 - 0s - 7ms/step - loss: 1.8027
Epoch 15/20
8/8 - 0s - 7ms/step - loss: 1.7550
Epoch 16/20
8/8 - 0s - 7ms/step - loss: 1.7105
Epoch 17/20
8/8 - 0s - 8ms/step - loss: 1.6642
Epoch 18/20
8/8 - 0s - 8ms/step - loss: 1.6243
Epoch 19/20
8/8 - 0s - 8ms/step - loss: 1.5804
Epoch 20/20
8/8 - 0s - 7ms/step - loss: 1.5391
2/2 - 1s - 346ms/step
[CV] END .............................l1_reg=0.1, n_units=20; total time=   4.1s
Epoch 1/20
10/10 - 2s - 218ms/step - loss: 1.2496
Epoch 2/20
10/10 - 0s - 8ms/step - loss: 1.1931
Epoch 3/20
10/10 - 0s - 3ms/step - loss: 1.1394
Epoch 4/20
10/10 - 0s - 5ms/step - loss: 1.0985
Epoch 5/20
10/10 - 0s - 4ms/step - loss: 1.0628
Epoch 6/20
10/10 - 0s - 4ms/step - loss: 1.0481
Epoch 7/20
10/10 - 0s - 4ms/step - loss: 1.0298
Epoch 8/20
10/10 - 0s - 4ms/step - loss: 1.0152
Epoch 9/20
10/10 - 0s - 5ms/step - loss: 1.0048
Epoch 10/20
10/10 - 0s - 9ms/step - loss: 0.9950
Epoch 11/20
10/10 - 0s - 5ms/step - loss: 0.9904
Epoch 12/20
10/10 - 0s - 6ms/step - loss: 0.9890
Epoch 13/20
10/10 - 0s - 6ms/step - loss: 0.9905
Epoch 14/20
10/10 - 0s - 5ms/step - loss: 0.9881
Epoch 15/20
10/10 - 0s - 6ms/step - loss: 0.9877
Epoch 16/20
10/10 - 0s - 7ms/step - loss: 0.9867
Epoch 17/20
10/10 - 0s - 6ms/step - loss: 0.9868
Epoch 18/20
10/10 - 0s - 7ms/step - loss: 0.9862
Epoch 19/20
10/10 - 0s - 6ms/step - loss: 0.9857
Epoch 20/20
10/10 - 0s - 6ms/step - loss: 0.9862
Best: 0.001173 using {'l1_reg': 0.01, 'n_units': 5}
nan (nan) with {'l1_reg': 0, 'n_units': 5}
nan (nan) with {'l1_reg': 0, 'n_units': 10}
nan (nan) with {'l1_reg': 0, 'n_units': 20}
-0.004313 (0.090929) with {'l1_reg': 0.001, 'n_units': 5}
-0.022342 (0.123261) with {'l1_reg': 0.001, 'n_units': 10}
-0.024614 (0.047997) with {'l1_reg': 0.001, 'n_units': 20}
0.001173 (0.067808) with {'l1_reg': 0.01, 'n_units': 5}
-0.020041 (0.116918) with {'l1_reg': 0.01, 'n_units': 10}
-0.028749 (0.046771) with {'l1_reg': 0.01, 'n_units': 20}
-0.017489 (0.088577) with {'l1_reg': 0.1, 'n_units': 5}
-0.017326 (0.081045) with {'l1_reg': 0.1, 'n_units': 10}
-0.028286 (0.032541) with {'l1_reg': 0.1, 'n_units': 20}

TRAIN CROSS-VALIDATED MODEL

In [64]:
if do_training is True:
    for key in params.keys():
        tf.random.set_seed(0)
        print('Training', key, 'model')
        model = params[key]['function'](params[key]['H'], params[key]['l1_reg'])
        model.fit(x_train, y_train, epochs=max_epochs, 
                  batch_size=batch_size, callbacks=[es], shuffle=False)
        params[key]['model'] = model

Training rnn model
Epoch 1/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - loss: 2.8958
Epoch 2/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 2.7985 
Epoch 3/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 2.7109 
Epoch 4/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 2.6285 
Epoch 5/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 2.5507 
Epoch 6/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 2.4775 
Epoch 7/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 2.4085 
Epoch 8/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 2.3434 
Epoch 9/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 2.2821 
Epoch 10/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 2.2241 
Epoch 11/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 2.1693 
Epoch 12/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 2.1174 
Epoch 13/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 2.0681 
Epoch 14/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 2.0213 
Epoch 15/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 1.9768 
Epoch 16/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 1.9344 
Epoch 17/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 1.8939 
Epoch 18/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 1.8553 
Epoch 19/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 1.8184 
Epoch 20/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 1.7831 
Restoring model weights from the end of the best epoch: 20.
Training gru model
Epoch 1/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 3s 4ms/step - loss: 2.0663
Epoch 2/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 1.7460 
Epoch 3/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 1.5141 
Epoch 4/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 1.3606 
Epoch 5/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 1.2813 
Epoch 6/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 1.2530 
Epoch 7/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 1.2397 
Epoch 8/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 1.2299 
Epoch 9/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 1.2234 
Epoch 10/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 1.2173 
Epoch 11/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 1.2112 
Epoch 12/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 1.2060 
Epoch 13/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 1.2017 
Epoch 14/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 1.1980 
Epoch 15/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 1.1946 
Epoch 16/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 1.1914 
Epoch 17/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 1.1884 
Epoch 18/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 1.1855 
Epoch 19/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 1.1827 
Epoch 20/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 1.1800 
Restoring model weights from the end of the best epoch: 20.
Training lstm model
Epoch 1/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 2s 3ms/step - loss: 1.4577
Epoch 2/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 1.3861 
Epoch 3/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 1.3206 
Epoch 4/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 1.2638 
Epoch 5/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 1.2194 
Epoch 6/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 1.1892 
Epoch 7/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 1.1719 
Epoch 8/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 1.1635 
Epoch 9/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 1.1596 
Epoch 10/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 1.1575 
Epoch 11/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 1.1562 
Epoch 12/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 1.1552 
Epoch 13/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 1.1542 
Epoch 14/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 1.1531 
Epoch 15/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 1.1520 
Epoch 16/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 1.1509 
Epoch 17/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - loss: 1.1498 
Epoch 18/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 1.1487 
Epoch 19/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 1.1477 
Epoch 20/20
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 1.1468 
Restoring model weights from the end of the best epoch: 20.
In [65]:
for key in params.keys():
    params[key]['model'].save('RNNs-IMPORTACIONES-VAFODO-SAVED-' + key + '.hdf5', overwrite=True)

WARNING:absl:You are saving your model as an HDF5 file via `model.save()` or `keras.saving.save_model(model)`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')` or `keras.saving.save_model(model, 'my_model.keras')`. 
WARNING:absl:You are saving your model as an HDF5 file via `model.save()` or `keras.saving.save_model(model)`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')` or `keras.saving.save_model(model, 'my_model.keras')`. 
WARNING:absl:You are saving your model as an HDF5 file via `model.save()` or `keras.saving.save_model(model)`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')` or `keras.saving.save_model(model, 'my_model.keras')`.

PREDICCION¶

In [66]:
for key in params.keys():
    model = params[key]['model']
    model.summary()
    
    params[key]['pred_train'] = model.predict(x_train, verbose=1)
    params[key]['MSE_train'] = mean_squared_error(y_train, params[key]['pred_train'])
    
    params[key]['pred_test'] = model.predict(x_test, verbose=1) 
    params[key]['MSE_test'] = mean_squared_error(y_test, params[key]['pred_test'])

Model: "sequential_183"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ Layer (type)                         ┃ Output Shape                ┃         Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ simple_rnn_61 (SimpleRNN)            │ (None, 5)                   │              35 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense_183 (Dense)                    │ (None, 1)                   │               6 │
└──────────────────────────────────────┴─────────────────────────────┴─────────────────┘

 Total params: 125 (504.00 B)

 Trainable params: 41 (164.00 B)

 Non-trainable params: 0 (0.00 B)

 Optimizer params: 84 (340.00 B)

3/3 ━━━━━━━━━━━━━━━━━━━━ 1s 232ms/step
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 56ms/step

Model: "sequential_184"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ Layer (type)                         ┃ Output Shape                ┃         Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ gru_61 (GRU)                         │ (None, 20)                  │           1,380 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense_184 (Dense)                    │ (None, 1)                   │              21 │
└──────────────────────────────────────┴─────────────────────────────┴─────────────────┘

 Total params: 4,205 (16.43 KB)

 Trainable params: 1,401 (5.47 KB)

 Non-trainable params: 0 (0.00 B)

 Optimizer params: 2,804 (10.96 KB)

3/3 ━━━━━━━━━━━━━━━━━━━━ 2s 481ms/step
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 39ms/step

Model: "sequential_185"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ Layer (type)                         ┃ Output Shape                ┃         Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ lstm_61 (LSTM)                       │ (None, 5)                   │             140 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense_185 (Dense)                    │ (None, 1)                   │               6 │
└──────────────────────────────────────┴─────────────────────────────┴─────────────────┘

 Total params: 440 (1.72 KB)

 Trainable params: 146 (584.00 B)

 Non-trainable params: 0 (0.00 B)

 Optimizer params: 294 (1.15 KB)

3/3 ━━━━━━━━━━━━━━━━━━━━ 1s 228ms/step
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 37ms/step

MODEL PERFORMANCE¶

In [67]:
print('training set:', len(y_train))
print('testing set:', len(y_test))

training set: 91
testing set: 5
In [68]:
print(params.keys())

dict_keys(['rnn', 'gru', 'lstm'])
In [69]:
import matplotlib.pyplot as plt
from datetime import timedelta

# Definir parámetros de visualización
max_pts = 10**4
compare = params.keys()  # Lista de modelos a comparar
l, u = (None, None)  # Índices inferior y superior para restringir el rango a graficar
ds = max(1, len(y_train[l:u]) // max_pts)  # Ratio de muestreo para limitar puntos a `max_pts` por serie

# Crear la figura con tamaño adecuado
fig, ax = plt.subplots(figsize=(15, 8))

# Extraer valores de tiempo y observaciones
x_vals = y_train_timestamps[l:u:ds]
observed_vals = y_train[l:u:ds]

# Graficar observaciones
ax.plot(x_vals, observed_vals, c="black", label="Observado", lw=1)

# Graficar predicciones de cada modelo
for key in compare:
    y_vals = params[key]['pred_train'][l:u:ds]
    label = f"{params[key]['label']} (MSE entrenamiento: {params[key]['MSE_train']:.2e})"
    ax.plot(x_vals, y_vals, c=params[key]['color'], label=label, lw=1)

# Configuración adicional y decoración
start, end = x_vals.min(), x_vals.max()
xticks = [start.date() + timedelta(days=i) for i in range((end - start).days + 1)]
xticks = xticks[::max(1, len(xticks) // 30)]

for t in xticks:
    ax.axvline(x=t, c='gray', linewidth=0.5, zorder=0)

ax.set_xticks(xticks)
ax.set_xticklabels([str(t) for t in xticks], rotation=70)
ax.set_xlim(start, end)

# Etiquetas y leyenda
ax.set_ylabel('$\hat{Y}$', rotation=0, fontsize=14)
ax.legend(loc="best", fontsize=12)

# Título del gráfico
ax.set_title('Observado vs Salidas del Modelo (Entrenamiento)', fontsize=16)

# Mostrar la gráfica
plt.show()
No description has been provided for this image
In [70]:
import matplotlib.pyplot as plt
from datetime import timedelta

# Definir parámetros de visualización
max_pts = 10**4
compare = params.keys()  # Lista de modelos a comparar
l, u = (None, None)  # Índices inferior y superior para restringir el rango a graficar
ds = max(1, len(y_test[l:u]) // max_pts)  # Ratio de muestreo para limitar puntos a `max_pts` por serie

# Crear la figura con tamaño adecuado
fig, ax = plt.subplots(figsize=(15, 8))

# Extraer valores de tiempo y observaciones
x_vals = y_test_timestamps[l:u:ds]
observed_vals = y_test[l:u:ds]

# Graficar observaciones
ax.plot(x_vals, observed_vals, c="black", label="Observado", lw=1)

# Graficar predicciones de cada modelo
for key in compare:
    y_vals = params[key]['pred_test'][l:u:ds]
    label = f"{params[key]['label']} (MSE prueba: {params[key]['MSE_test']:.2e})"
    ax.plot(x_vals, y_vals, c=params[key]['color'], label=label, lw=1)

# Configuración adicional y decoración
start, end = x_vals.min(), x_vals.max()
xticks = [start.date() + timedelta(days=i) for i in range((end - start).days + 1)]
xticks = xticks[::max(1, len(xticks) // 30)]

for t in xticks:
    ax.axvline(x=t, c='gray', linewidth=0.5, zorder=0)

ax.set_xticks(xticks)
ax.set_xticklabels([str(t) for t in xticks], rotation=70)
ax.set_xlim(start, end)

# Etiquetas y leyenda
ax.set_ylabel('$\hat{Y}$', rotation=0, fontsize=14)
ax.legend(loc="best", fontsize=12)

# Título del gráfico
ax.set_title('Observado vs Salidas del Modelo (Pruebas)', fontsize=16)

# Mostrar la gráfica
plt.show()
No description has been provided for this image
In [71]:
import matplotlib.pyplot as plt
from datetime import timedelta

# Definir parámetros de visualización
max_pts = 10**4
compare = params.keys()  # Lista de modelos a comparar
l, u = (None, None)  # Índices inferior y superior para restringir el rango a graficar
ds = max(1, len(y_test[l:u]) // max_pts)  # Ratio de muestreo para limitar puntos a `max_pts` por serie

# Crear la figura con tamaño adecuado
fig, ax = plt.subplots(figsize=(15, 8))

# Extraer valores de tiempo y errores de predicción
x_vals = y_test_timestamps[l:u:ds]
for key in compare:
    y_vals = params[key]['pred_test'][l:u:ds] - y_test[l:u:ds]
    label = f"{params[key]['label']} (MSE prueba: {params[key]['MSE_test']:.2e})"
    ax.plot(x_vals, y_vals, c=params[key]['color'], label=label, lw=1)

# Configuración adicional y decoración
start, end = x_vals.min(), x_vals.max()
xticks = [start.date() + timedelta(days=i) for i in range((end - start).days + 1)]
xticks = xticks[::max(1, len(xticks) // 30)]

ax.axhline(0, linewidth=0.8, color='black', linestyle='--')  # Línea horizontal en y=0 para referencia

for t in xticks:
    ax.axvline(x=t, c='gray', linewidth=0.5, zorder=0)

ax.set_xticks(xticks)
ax.set_xticklabels([str(t) for t in xticks], rotation=80)
ax.set_xlim(start, end)

# Etiquetas y leyenda
ax.set_ylabel('$\hat{Y}-Y$', fontsize=14)
ax.legend(loc="best", fontsize=12)

# Título del gráfico
ax.set_title('Observado vs Error del Modelo (Pruebas)', fontsize=16)

# Mostrar la gráfica
plt.show()
No description has been provided for this image
In [72]:
import matplotlib.pyplot as plt
from datetime import timedelta

# Definir parámetros de visualización
max_pts = 10**4
compare = params.keys()  # Lista de modelos a comparar
l, u = (None, None)  # Índices inferior y superior para restringir el rango a graficar
ds = max(1, len(y_train[l:u]) // max_pts)  # Ratio de muestreo para limitar puntos a `max_pts` por serie

# Crear la figura con tamaño adecuado
fig, ax = plt.subplots(figsize=(15, 8))

# Extraer valores de tiempo y errores de predicción
x_vals = y_train_timestamps[l:u:ds]
for key in compare:
    y_vals = params[key]['pred_train'][l:u:ds] - y_train[l:u:ds]
    label = f"{params[key]['label']} (MSE entrenamiento: {params[key]['MSE_train']:.2e})"
    ax.plot(x_vals, y_vals, c=params[key]['color'], label=label, lw=1)

# Configuración adicional y decoración
start, end = x_vals.min(), x_vals.max()
xticks = [start.date() + timedelta(days=i) for i in range((end - start).days + 1)]
xticks = xticks[::max(1, len(xticks) // 30)]

ax.axhline(0, linewidth=0.8, color='black', linestyle='--')  # Línea horizontal en y=0 para referencia

for t in xticks:
    ax.axvline(x=t, c='gray', linewidth=0.5, zorder=0)

ax.set_xticks(xticks)
ax.set_xticklabels([str(t) for t in xticks], rotation=80)
ax.set_xlim(start, end)

# Etiquetas y leyenda
ax.set_ylabel('$\hat{Y}-Y$', fontsize=14)
ax.legend(loc="best", fontsize=12)

# Título del gráfico
ax.set_title('Observado vs Error del Modelo (Entrenamiento)', fontsize=16)

# Mostrar la gráfica
plt.show()
No description has been provided for this image